{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/theamrzaki/text_summurization_abstractive_methods/blob/master/Model_3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-tk5ROC6mw0f"
   },
   "source": [
    "# Text Summarization of Amazon reviews\n",
    "\n",
    "In this notebook I will write summaries with the help of my Seq2Seq model in Summarizer.py.\n",
    "\n",
    "The model works impressively well in the end!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "B_ULQF2smw0h"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import tensorflow_hub as hub\n",
    "from collections import Counter\n",
    "\n",
    "#import Summarizer\n",
    "#import summarizer_data_utils\n",
    "#import summarizer_model_utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "d1na5Zz5A-8S",
    "outputId": "90018806-ca07-4e57-9632-7b4a46b9135f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.12.0-rc2\n"
     ]
    }
   ],
   "source": [
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BcC4F_T1BPq8"
   },
   "source": [
    "## Helpers (Google Connect, Summurizer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OrsJFD6KBsV7"
   },
   "source": [
    "### Google Connect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0k500ZKFBSOt"
   },
   "outputs": [],
   "source": [
    "#working google drive ya RAB\n",
    "#https://stackoverflow.com/questions/52385655/unable-to-locate-package-google-drive-ocamlfuse-suddenly-stopped-working\n",
    "\n",
    "\n",
    "!apt-get install -y -qq software-properties-common python-software-properties module-init-tools\n",
    "!wget https://launchpad.net/~alessandro-strada/+archive/ubuntu/google-drive-ocamlfuse-beta/+build/15331130/+files/google-drive-ocamlfuse_0.7.0-0ubuntu1_amd64.deb\n",
    "!dpkg -i google-drive-ocamlfuse_0.7.0-0ubuntu1_amd64.deb\n",
    "!apt-get install -f\n",
    "!apt-get -y install -qq fuse\n",
    "from google.colab import auth\n",
    "auth.authenticate_user()\n",
    "from oauth2client.client import GoogleCredentials\n",
    "creds = GoogleCredentials.get_application_default()\n",
    "import getpass\n",
    "!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL\n",
    "vcode = getpass.getpass()\n",
    "!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}\n",
    "\n",
    "!mkdir -p drive\n",
    "!google-drive-ocamlfuse drive"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8ByAY0LEBxRr"
   },
   "source": [
    "### Summurizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "F_jzv5I0B19T"
   },
   "source": [
    "#### summarizer_model_utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_xWv67GaBzJ1"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from nltk.translate.bleu_score import sentence_bleu\n",
    "\n",
    "\n",
    "def minibatches(inputs, targets, minibatch_size):\n",
    "    \"\"\"batch generator. yields x and y batch.\n",
    "    \"\"\"\n",
    "    x_batch, y_batch = [], []\n",
    "    for inp, tgt in zip(inputs, targets):\n",
    "        if len(x_batch) == minibatch_size and len(y_batch) == minibatch_size:\n",
    "            yield x_batch, y_batch\n",
    "            x_batch, y_batch = [], []\n",
    "        x_batch.append(inp)\n",
    "        y_batch.append(tgt)\n",
    "\n",
    "    if len(x_batch) != 0:\n",
    "        for inp, tgt in zip(inputs, targets):\n",
    "            if len(x_batch) != minibatch_size:\n",
    "                x_batch.append(inp)\n",
    "                y_batch.append(tgt)\n",
    "            else:\n",
    "                break\n",
    "        yield x_batch, y_batch\n",
    "\n",
    "\n",
    "def pad_sequences(sequences, pad_tok, tail=True):\n",
    "    \"\"\"Pads the sentences, so that all sentences in a batch have the same length.\n",
    "    \"\"\"\n",
    "\n",
    "    max_length = max(len(x) for x in sequences)\n",
    "\n",
    "    sequence_padded, sequence_length = [], []\n",
    "\n",
    "    for seq in sequences:\n",
    "        seq = list(seq)\n",
    "        if tail:\n",
    "            seq_ = seq[:max_length] + [pad_tok] * max(max_length - len(seq), 0)\n",
    "        else:\n",
    "            seq_ = [pad_tok] * max(max_length - len(seq), 0) + seq[:max_length]\n",
    "\n",
    "        sequence_padded += [seq_]\n",
    "        sequence_length += [min(len(seq), max_length)]\n",
    "\n",
    "    return sequence_padded, sequence_length\n",
    "\n",
    "\n",
    "def sample_results(preds, ind2word, word2ind, converted_summaries, converted_texts, use_bleu=False):\n",
    "    \"\"\"Plots the actual text and summary and the corresponding created summary.\n",
    "    takes care of whether beam search or greedy decoder was used.\n",
    "    \"\"\"\n",
    "    beam = False\n",
    "\n",
    "    if len(np.array(preds).shape) == 4:\n",
    "        beam = True\n",
    "\n",
    "    '''Bleu score is not used correctly here, but serves as reference.\n",
    "    '''\n",
    "    if use_bleu:\n",
    "        bleu_scores = []\n",
    "\n",
    "    for pred, summary, text, seq_length in zip(preds[0],\n",
    "                                               converted_summaries,\n",
    "                                               converted_texts,\n",
    "                                               [len(inds) for inds in converted_summaries]):\n",
    "        print('\\n\\n\\n', 100 * '-')\n",
    "        if beam:\n",
    "            actual_text = [ind2word[word] for word in text if\n",
    "                           word != word2ind[\"<SOS>\"] and word != word2ind[\"<EOS>\"]]\n",
    "            actual_summary = [ind2word[word] for word in summary if\n",
    "                              word != word2ind['<EOS>'] and word != word2ind['<SOS>']]\n",
    "\n",
    "            created_summary = []\n",
    "            for word in pred:\n",
    "                if word[0] != word2ind['<SOS>'] and word[0] != word2ind['<EOS>']:\n",
    "                    created_summary.append(ind2word[word[0]])\n",
    "                    continue\n",
    "                else:\n",
    "                    continue\n",
    "\n",
    "            print('Actual Text:\\n{}\\n'.format(' '.join(actual_text)))\n",
    "            print('Actual Summary:\\n{}\\n'.format(' '.join(actual_summary)))\n",
    "            print('Created Summary:\\n{}\\n'.format(' '.join(created_summary)))\n",
    "            if use_bleu:\n",
    "                bleu_score = sentence_bleu([actual_summary], created_summary)\n",
    "                bleu_scores.append(bleu_score)\n",
    "                print('Bleu-score:', bleu_score)\n",
    "\n",
    "            print()\n",
    "\n",
    "\n",
    "        else:\n",
    "            actual_text = [ind2word[word] for word in text if\n",
    "                           word != word2ind[\"<SOS>\"] and word != word2ind[\"<EOS>\"]]\n",
    "            actual_summary = [ind2word[word] for word in summary if\n",
    "                              word != word2ind['<EOS>'] and word != word2ind['<SOS>']]\n",
    "            created_summary = [ind2word[word] for word in pred if\n",
    "                               word != word2ind['<EOS>'] and word != word2ind['<SOS>']]\n",
    "\n",
    "            print('Actual Text:\\n{}\\n'.format(' '.join(actual_text)))\n",
    "            print('Actual Summary:\\n{}\\n'.format(' '.join(actual_summary)))\n",
    "            print('Created Summary:\\n{}\\n'.format(' '.join(created_summary)))\n",
    "            if use_bleu:\n",
    "                bleu_score = sentence_bleu([actual_summary], created_summary)\n",
    "                bleu_scores.append(bleu_score)\n",
    "                print('Bleu-score:', bleu_score)\n",
    "\n",
    "    if use_bleu:\n",
    "        bleu_score = np.mean(bleu_scores)\n",
    "        print('\\n\\n\\nTotal Bleu Score:', bleu_score)\n",
    "\n",
    "\n",
    "def reset_graph(seed=97):\n",
    "    \"\"\"helper function to reset the default graph. this often\n",
    "       comes handy when using jupyter noteboooks.\n",
    "    \"\"\"\n",
    "    tf.reset_default_graph()\n",
    "    tf.set_random_seed(seed)\n",
    "    np.random.seed(seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WijZf_vWB3-z"
   },
   "source": [
    "#### summarizer_data_utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "c6PGK7hBB8ZD"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "import re\n",
    "import html\n",
    "from collections import Counter\n",
    "\n",
    "import nltk\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "def preprocess_sentence(text, keep_most=False):\n",
    "    \"\"\"\n",
    "    Helper function to remove html, unneccessary spaces and punctuation.\n",
    "    Args:\n",
    "        text: String.\n",
    "        keep_most: Boolean. depending if True or False, we either\n",
    "                   keep only letters and numbers or also other characters.\n",
    "    Returns:\n",
    "        processed text.\n",
    "    \"\"\"\n",
    "    text = text.lower()\n",
    "    text = fixup(text)\n",
    "    text = re.sub(r\"<br />\", \" \", text)\n",
    "    if keep_most:\n",
    "        text = re.sub(r\"[^a-z0-9%!?.,:()/]\", \" \", text)\n",
    "    else:\n",
    "        text = re.sub(r\"[^a-z0-9]\", \" \", text)\n",
    "    text = re.sub(r\"    \", \" \", text)\n",
    "    text = re.sub(r\"   \", \" \", text)\n",
    "    text = re.sub(r\"  \", \" \", text)\n",
    "    text = text.strip()\n",
    "    return text\n",
    "\n",
    "\n",
    "def fixup(x):\n",
    "    re1 = re.compile(r'  +')\n",
    "    x = x.replace('#39;', \"'\").replace('amp;', '&').replace('#146;', \"'\").replace(\n",
    "        'nbsp;', ' ').replace('#36;', '$').replace('\\\\n', \"\\n\").replace('quot;', \"'\").replace(\n",
    "        '<br />', \"\\n\").replace('\\\\\"', '\"').replace('<unk>', 'u_n').replace(' @.@ ', '.').replace(\n",
    "        ' @-@ ', '-').replace('\\\\', ' \\\\ ')\n",
    "    return re1.sub(' ', html.unescape(x))\n",
    "\n",
    "\n",
    "def preprocess(text, keep_most=False):\n",
    "    \"\"\"\n",
    "    Splits the text into sentences, preprocesses\n",
    "       and tokenizes each sentence.\n",
    "    Args:\n",
    "        text: String. multiple sentences.\n",
    "        keep_most: Boolean. depending if True or False, we either\n",
    "                   keep only letters and numbers or also other characters.\n",
    "    Returns:\n",
    "        preprocessed and tokenized text.\n",
    "    \"\"\"\n",
    "    tokenized = []\n",
    "    for sentence in nltk.sent_tokenize(text):\n",
    "        sentence = preprocess_sentence(sentence, keep_most)\n",
    "        sentence = nltk.word_tokenize(sentence)\n",
    "        for token in sentence:\n",
    "            tokenized.append(token)\n",
    "\n",
    "    return tokenized\n",
    "\n",
    "\n",
    "def preprocess_texts_and_summaries(texts,\n",
    "                                   summaries,\n",
    "                                   keep_most=False):\n",
    "    \"\"\"iterates given list of texts and given list of summaries and tokenizes every\n",
    "       review using the tokenize_review() function.\n",
    "       apart from that we count up all the words in the texts and summaries.\n",
    "       returns: - processed texts\n",
    "                - processed summaries\n",
    "                - array containing all the unique words together with their counts\n",
    "                  sorted by counts.\n",
    "    \"\"\"\n",
    "\n",
    "    start_time = time.time()\n",
    "    processed_texts = []\n",
    "    processed_summaries = []\n",
    "    words = []\n",
    "\n",
    "    for text in texts:\n",
    "        text = preprocess(text, keep_most)\n",
    "        for word in text:\n",
    "            words.append(word)\n",
    "        processed_texts.append(text)\n",
    "    for summary in summaries:\n",
    "        summary = preprocess(summary, keep_most)\n",
    "        for word in summary:\n",
    "            words.append(word)\n",
    "\n",
    "        processed_summaries.append(summary)\n",
    "    words_counted = Counter(words).most_common()\n",
    "    print('Processing Time: ', time.time() - start_time)\n",
    "\n",
    "    return processed_texts, processed_summaries, words_counted\n",
    "\n",
    "\n",
    "def create_word_inds_dicts(words_counted,\n",
    "                           specials=None,\n",
    "                           min_occurences=0):\n",
    "    \"\"\" creates lookup dicts from word to index and back.\n",
    "        returns the lookup dicts and an array of words that were not used,\n",
    "        due to rare occurence.\n",
    "    \"\"\"\n",
    "    missing_words = []\n",
    "    word2ind = {}\n",
    "    ind2word = {}\n",
    "    i = 0\n",
    "\n",
    "    if specials is not None:\n",
    "        for sp in specials:\n",
    "            word2ind[sp] = i\n",
    "            ind2word[i] = sp\n",
    "            i += 1\n",
    "\n",
    "    for (word, count) in words_counted:\n",
    "        if count >= min_occurences:\n",
    "            word2ind[word] = i\n",
    "            ind2word[i] = word\n",
    "            i += 1\n",
    "        else:\n",
    "            missing_words.append(word)\n",
    "\n",
    "    return word2ind, ind2word, missing_words\n",
    "\n",
    "\n",
    "def convert_sentence(review, word2ind):\n",
    "    \"\"\" converts the given sent to int values corresponding to the given word2ind\"\"\"\n",
    "    inds = []\n",
    "    unknown_words = []\n",
    "\n",
    "    for word in review:\n",
    "        if word in word2ind.keys():\n",
    "            inds.append(int(word2ind[word]))\n",
    "        else:\n",
    "            inds.append(int(word2ind['<UNK>']))\n",
    "            unknown_words.append(word)\n",
    "\n",
    "    return inds, unknown_words\n",
    "\n",
    "\n",
    "def convert_to_inds(input, word2ind, eos=False, sos=False):\n",
    "    converted_input = []\n",
    "    all_unknown_words = set()\n",
    "\n",
    "    for inp in input:\n",
    "        converted_inp, unknown_words = convert_sentence(inp, word2ind)\n",
    "        if eos:\n",
    "            converted_inp.append(word2ind['<EOS>'])\n",
    "        if sos:\n",
    "            converted_inp.insert(0, word2ind['<SOS>'])\n",
    "        converted_input.append(converted_inp)\n",
    "        all_unknown_words.update(unknown_words)\n",
    "\n",
    "    return converted_input, all_unknown_words\n",
    "\n",
    "\n",
    "def convert_inds_to_text(inds, ind2word, preprocess=False):\n",
    "    \"\"\" convert the given indexes back to text \"\"\"\n",
    "    words = [ind2word[word] for word in inds]\n",
    "    return words\n",
    "\n",
    "\n",
    "def load_pretrained_embeddings(path):\n",
    "    \"\"\"loads pretrained embeddings. stores each embedding in a\n",
    "       dictionary with its corresponding word\n",
    "    \"\"\"\n",
    "    embeddings = {}\n",
    "    with open(path, 'r', encoding='utf-8') as f:\n",
    "        for line in f:\n",
    "            values = line.split(' ')\n",
    "            word = values[0]\n",
    "            embedding_vector = np.array(values[1:], dtype='float32')\n",
    "            embeddings[word] = embedding_vector\n",
    "    return embeddings\n",
    "\n",
    "\n",
    "def create_and_save_embedding_matrix(word2ind,\n",
    "                                     pretrained_embeddings_path,\n",
    "                                     save_path,\n",
    "                                     embedding_dim=300):\n",
    "    \"\"\"creates embedding matrix for each word in word2ind. if that words is in\n",
    "       pretrained_embeddings, that vector is used. otherwise initialized\n",
    "       randomly.\n",
    "    \"\"\"\n",
    "    pretrained_embeddings = load_pretrained_embeddings(pretrained_embeddings_path)\n",
    "    embedding_matrix = np.zeros((len(word2ind), embedding_dim), dtype=np.float32)\n",
    "    for word, i in word2ind.items():\n",
    "        if word in pretrained_embeddings.keys():\n",
    "            embedding_matrix[i] = pretrained_embeddings[word]\n",
    "        else:\n",
    "            embedding = np.array(np.random.uniform(-1.0, 1.0, embedding_dim))\n",
    "            embedding_matrix[i] = embedding\n",
    "    if not os.path.exists(os.path.dirname(save_path)):\n",
    "        os.makedirs(os.path.dirname(save_path))\n",
    "    np.save(save_path, embedding_matrix)\n",
    "    return np.array(embedding_matrix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GXGQKvuFCCIN"
   },
   "source": [
    "#### Summarizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "J6k3dNSHCCjL"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow.python.layers.core import Dense\n",
    "\n",
    "#import summarizer_model_utils\n",
    "\n",
    "\n",
    "class Summarizer:\n",
    "\n",
    "    def __init__(self,\n",
    "                 word2ind,\n",
    "                 ind2word,\n",
    "                 save_path,\n",
    "                 mode='TRAIN',\n",
    "                 num_layers_encoder=1,\n",
    "                 num_layers_decoder=1,\n",
    "                 embedding_dim=300,\n",
    "                 rnn_size_encoder=256,\n",
    "                 rnn_size_decoder=256,\n",
    "                 learning_rate=0.001,\n",
    "                 learning_rate_decay=0.9,\n",
    "                 learning_rate_decay_steps=100,\n",
    "                 max_lr=0.01,\n",
    "                 keep_probability=0.8,\n",
    "                 batch_size=64,\n",
    "                 beam_width=10,\n",
    "                 epochs=20,\n",
    "                 eos=\"<EOS>\",\n",
    "                 sos=\"<SOS>\",\n",
    "                 pad='<PAD>',\n",
    "                 clip=5,\n",
    "                 inference_targets=False,\n",
    "                 pretrained_embeddings_path=None,\n",
    "                 summary_dir=None,\n",
    "                 use_cyclic_lr=False):\n",
    "        \"\"\"\n",
    "        Args:\n",
    "            word2ind: lookup dict from word to index.\n",
    "            ind2word: lookup dict from index to word.\n",
    "            save_path: path to save the tf model to in the end.\n",
    "            mode: String. 'TRAIN' or 'INFER'. depending on which mode we use\n",
    "                  a different graph is created.\n",
    "            num_layers_encoder: Float. Number of encoder layers. defaults to 1.\n",
    "            num_layers_decoder: Float. Number of decoder layers. defaults to 1.\n",
    "            embedding_dim: dimension of the embedding vectors in the embedding matrix.\n",
    "                           every word has a embedding_dim 'long' vector.\n",
    "            rnn_size_encoder: Integer. number of hidden units in encoder. defaults to 256.\n",
    "            rnn_size_decoder: Integer. number of hidden units in decoder. defaults to 256.\n",
    "            learning_rate: Float.\n",
    "            learning_rate_decay: only if exponential learning rate is used.\n",
    "            learning_rate_decay_steps: Integer.\n",
    "            max_lr: only used if cyclic learning rate is used.\n",
    "            keep_probability: Float.\n",
    "            batch_size: Integer. Size of minibatches.\n",
    "            beam_width: Integer. Only used in inference, for Beam Search.('INFER'-mode)\n",
    "            epochs: Integer. Number of times the training is conducted\n",
    "                    on the whole training data.\n",
    "            eos: EndOfSentence tag.\n",
    "            sos: StartOfSentence tag.\n",
    "            pad: Padding tag.\n",
    "            clip: Value to clip the gradients to in training process.\n",
    "            inference_targets:\n",
    "            pretrained_embeddings_path: Path to pretrained embeddings. Has to be .npy\n",
    "            summary_dir: Directory the summaries are written to for tensorboard.\n",
    "            use_cyclic_lr: Boolean.\n",
    "        \"\"\"\n",
    "\n",
    "        self.word2ind = word2ind\n",
    "        self.ind2word = ind2word\n",
    "        self.vocab_size = len(word2ind)\n",
    "        self.num_layers_encoder = num_layers_encoder\n",
    "        self.num_layers_decoder = num_layers_decoder\n",
    "        self.rnn_size_encoder = rnn_size_encoder\n",
    "        self.rnn_size_decoder = rnn_size_decoder\n",
    "        self.save_path = save_path\n",
    "        self.embedding_dim = embedding_dim\n",
    "        self.mode = mode.upper()\n",
    "        self.learning_rate = learning_rate\n",
    "        self.learning_rate_decay = learning_rate_decay\n",
    "        self.learning_rate_decay_steps = learning_rate_decay_steps\n",
    "        self.keep_probability = keep_probability\n",
    "        self.batch_size = batch_size\n",
    "        self.beam_width = beam_width\n",
    "        self.eos = eos\n",
    "        self.sos = sos\n",
    "        self.clip = clip\n",
    "        self.pad = pad\n",
    "        self.epochs = epochs\n",
    "        self.inference_targets = inference_targets\n",
    "        self.pretrained_embeddings_path = pretrained_embeddings_path\n",
    "        self.use_cyclic_lr = use_cyclic_lr\n",
    "        self.max_lr = max_lr\n",
    "        self.summary_dir = summary_dir\n",
    "\n",
    "    def build_graph(self):\n",
    "        self.add_placeholders()\n",
    "        self.add_embeddings()\n",
    "        self.add_lookup_ops()\n",
    "        self.initialize_session()\n",
    "        self.add_seq2seq()\n",
    "        self.saver = tf.train.Saver()\n",
    "        print('Graph built.')\n",
    "\n",
    "    def add_placeholders(self):\n",
    "        self.ids_1 = tf.placeholder(tf.int32,\n",
    "                                    shape=[None, None],\n",
    "                                    name='ids_source')\n",
    "        self.ids_2 = tf.placeholder(tf.int32,\n",
    "                                    shape=[None, None],\n",
    "                                    name='ids_target')\n",
    "        self.sequence_lengths_1 = tf.placeholder(tf.int32,\n",
    "                                                 shape=[None],\n",
    "                                                 name='sequence_length_source')\n",
    "        self.sequence_lengths_2 = tf.placeholder(tf.int32,\n",
    "                                                 shape=[None],\n",
    "                                                 name='sequence_length_target')\n",
    "        self.maximum_iterations = tf.reduce_max(self.sequence_lengths_2,\n",
    "                                                name='max_dec_len')\n",
    "\n",
    "    def create_word_embedding(self, embed_name, vocab_size, embed_dim):\n",
    "        \"\"\"Creates embedding matrix in given shape - [vocab_size, embed_dim].\n",
    "        \"\"\"\n",
    "        embedding = tf.get_variable(embed_name,\n",
    "                                    shape=[vocab_size, embed_dim],\n",
    "                                    dtype=tf.float32)\n",
    "        return embedding\n",
    "\n",
    "    def add_embeddings(self):\n",
    "        \"\"\"Creates the embedding matrix. In case path to pretrained embeddings is given,\n",
    "           that embedding is loaded. Otherwise created.\n",
    "        \"\"\"\n",
    "        if self.pretrained_embeddings_path is not None:\n",
    "            self.embedding = tf.Variable(np.load(self.pretrained_embeddings_path),\n",
    "                                         name='embedding')\n",
    "            print('Loaded pretrained embeddings.')\n",
    "        else:\n",
    "            self.embedding = self.create_word_embedding('embedding',\n",
    "                                                        self.vocab_size,\n",
    "                                                        self.embedding_dim)\n",
    "\n",
    "    def add_lookup_ops(self):\n",
    "        \"\"\"Additional lookup operation for both source embedding and target embedding matrix.\n",
    "        \"\"\"\n",
    "        self.word_embeddings_1 = tf.nn.embedding_lookup(self.embedding,\n",
    "                                                        self.ids_1,\n",
    "                                                        name='word_embeddings_1')\n",
    "        self.word_embeddings_2 = tf.nn.embedding_lookup(self.embedding,\n",
    "                                                        self.ids_2,\n",
    "                                                        name='word_embeddings_2')\n",
    "\n",
    "    def make_rnn_cell(self, rnn_size, keep_probability):\n",
    "        \"\"\"Creates LSTM cell wrapped with dropout.\n",
    "        \"\"\"\n",
    "        cell = tf.nn.rnn_cell.LSTMCell(rnn_size)\n",
    "        cell = tf.nn.rnn_cell.DropoutWrapper(cell, input_keep_prob=keep_probability)\n",
    "        return cell\n",
    "\n",
    "    def make_attention_cell(self, dec_cell, rnn_size, enc_output, lengths, alignment_history=False):\n",
    "        \"\"\"Wraps the given cell with Bahdanau Attention.\n",
    "        \"\"\"\n",
    "        attention_mechanism = tf.contrib.seq2seq.BahdanauAttention(num_units=rnn_size,\n",
    "                                                                   memory=enc_output,\n",
    "                                                                   memory_sequence_length=lengths,\n",
    "                                                                   name='BahdanauAttention')\n",
    "\n",
    "        return tf.contrib.seq2seq.AttentionWrapper(cell=dec_cell,\n",
    "                                                   attention_mechanism=attention_mechanism,\n",
    "                                                   attention_layer_size=None,\n",
    "                                                   output_attention=False,\n",
    "                                                   alignment_history=alignment_history)\n",
    "\n",
    "    def triangular_lr(self, current_step):\n",
    "        \"\"\"cyclic learning rate - exponential range.\"\"\"\n",
    "        step_size = self.learning_rate_decay_steps\n",
    "        base_lr = self.learning_rate\n",
    "        max_lr = self.max_lr\n",
    "\n",
    "        cycle = tf.floor(1 + current_step / (2 * step_size))\n",
    "        x = tf.abs(current_step / step_size - 2 * cycle + 1)\n",
    "        lr = base_lr + (max_lr - base_lr) * tf.maximum(0.0, tf.cast((1.0 - x), dtype=tf.float32)) * (0.99999 ** tf.cast(\n",
    "            current_step,\n",
    "            dtype=tf.float32))\n",
    "        return lr\n",
    "\n",
    "\n",
    "    def add_seq2seq(self):\n",
    "        \"\"\"Creates the sequence to sequence architecture.\"\"\"\n",
    "        with tf.variable_scope('dynamic_seq2seq', dtype=tf.float32):\n",
    "            # Encoder\n",
    "            encoder_outputs, encoder_state = self.build_encoder()\n",
    "\n",
    "            # Decoder\n",
    "            logits, sample_id, final_context_state = self.build_decoder(encoder_outputs,\n",
    "                                                                        encoder_state)\n",
    "            if self.mode == 'TRAIN':\n",
    "\n",
    "                # Loss\n",
    "                loss = self.compute_loss(logits)\n",
    "                self.train_loss = loss\n",
    "                self.eval_loss = loss\n",
    "                self.global_step = tf.Variable(0, trainable=False)\n",
    "\n",
    "\n",
    "                # cyclic learning rate\n",
    "                if self.use_cyclic_lr:\n",
    "                    self.learning_rate = self.triangular_lr(self.global_step)\n",
    "\n",
    "                # exponential learning rate\n",
    "                else:\n",
    "                    self.learning_rate = tf.train.exponential_decay(\n",
    "                        self.learning_rate,\n",
    "                        self.global_step,\n",
    "                        decay_steps=self.learning_rate_decay_steps,\n",
    "                        decay_rate=self.learning_rate_decay,\n",
    "                        staircase=True)\n",
    "\n",
    "                # Optimizer\n",
    "                opt = tf.train.AdamOptimizer(self.learning_rate)\n",
    "\n",
    "\n",
    "                # Gradients\n",
    "                if self.clip > 0:\n",
    "                    grads, vs = zip(*opt.compute_gradients(self.train_loss))\n",
    "                    grads, _ = tf.clip_by_global_norm(grads, self.clip)\n",
    "                    self.train_op = opt.apply_gradients(zip(grads, vs),\n",
    "                                                        global_step=self.global_step)\n",
    "                else:\n",
    "                    self.train_op = opt.minimize(self.train_loss,\n",
    "                                                 global_step=self.global_step)\n",
    "\n",
    "\n",
    "\n",
    "            elif self.mode == 'INFER':\n",
    "                loss = None\n",
    "                self.infer_logits, _, self.final_context_state, self.sample_id = logits, loss, final_context_state, sample_id\n",
    "                self.sample_words = self.sample_id\n",
    "\n",
    "    def build_encoder(self):\n",
    "        \"\"\"The encoder. Bidirectional LSTM.\"\"\"\n",
    "\n",
    "        with tf.variable_scope(\"encoder\"):\n",
    "            fw_cell = self.make_rnn_cell(self.rnn_size_encoder // 2, self.keep_probability)\n",
    "            bw_cell = self.make_rnn_cell(self.rnn_size_encoder // 2, self.keep_probability)\n",
    "\n",
    "            for _ in range(self.num_layers_encoder):\n",
    "                (out_fw, out_bw), (state_fw, state_bw) = tf.nn.bidirectional_dynamic_rnn(\n",
    "                    cell_fw=fw_cell,\n",
    "                    cell_bw=bw_cell,\n",
    "                    inputs=self.word_embeddings_1,\n",
    "                    sequence_length=self.sequence_lengths_1,\n",
    "                    dtype=tf.float32)\n",
    "                encoder_outputs = tf.concat((out_fw, out_bw), -1)\n",
    "\n",
    "            bi_state_c = tf.concat((state_fw.c, state_bw.c), -1)\n",
    "            bi_state_h = tf.concat((state_fw.h, state_bw.h), -1)\n",
    "            bi_lstm_state = tf.nn.rnn_cell.LSTMStateTuple(c=bi_state_c, h=bi_state_h)\n",
    "            encoder_state = tuple([bi_lstm_state] * self.num_layers_encoder)\n",
    "\n",
    "            return encoder_outputs, encoder_state\n",
    "\n",
    "\n",
    "    def build_decoder(self, encoder_outputs, encoder_state):\n",
    "\n",
    "        sos_id_2 = tf.cast(self.word2ind[self.sos], tf.int32)\n",
    "        eos_id_2 = tf.cast(self.word2ind[self.eos], tf.int32)\n",
    "        self.output_layer = Dense(self.vocab_size, name='output_projection')\n",
    "\n",
    "        # Decoder.\n",
    "        with tf.variable_scope(\"decoder\") as decoder_scope:\n",
    "\n",
    "            cell, decoder_initial_state = self.build_decoder_cell(\n",
    "                encoder_outputs,\n",
    "                encoder_state,\n",
    "                self.sequence_lengths_1)\n",
    "\n",
    "            # Train\n",
    "            if self.mode != 'INFER':\n",
    "\n",
    "                helper = tf.contrib.seq2seq.ScheduledEmbeddingTrainingHelper(\n",
    "                    inputs=self.word_embeddings_2,\n",
    "                    sequence_length=self.sequence_lengths_2,\n",
    "                    embedding=self.embedding,\n",
    "                    sampling_probability=0.5,\n",
    "                    time_major=False)\n",
    "\n",
    "                # Decoder\n",
    "                my_decoder = tf.contrib.seq2seq.BasicDecoder(cell,\n",
    "                                                             helper,\n",
    "                                                             decoder_initial_state,\n",
    "                                                             output_layer=self.output_layer)\n",
    "\n",
    "                # Dynamic decoding\n",
    "                outputs, final_context_state, _ = tf.contrib.seq2seq.dynamic_decode(\n",
    "                    my_decoder,\n",
    "                    output_time_major=False,\n",
    "                    maximum_iterations=self.maximum_iterations,\n",
    "                    swap_memory=False,\n",
    "                    impute_finished=True,\n",
    "                    scope=decoder_scope\n",
    "                )\n",
    "\n",
    "                sample_id = outputs.sample_id\n",
    "                logits = outputs.rnn_output\n",
    "\n",
    "\n",
    "            # Inference\n",
    "            else:\n",
    "                start_tokens = tf.fill([self.batch_size], sos_id_2)\n",
    "                end_token = eos_id_2\n",
    "\n",
    "                # beam search\n",
    "                if self.beam_width > 0:\n",
    "                    my_decoder = tf.contrib.seq2seq.BeamSearchDecoder(\n",
    "                        cell=cell,\n",
    "                        embedding=self.embedding,\n",
    "                        start_tokens=start_tokens,\n",
    "                        end_token=end_token,\n",
    "                        initial_state=decoder_initial_state,\n",
    "                        beam_width=self.beam_width,\n",
    "                        output_layer=self.output_layer,\n",
    "                    )\n",
    "\n",
    "                # greedy\n",
    "                else:\n",
    "                    helper = tf.contrib.seq2seq.GreedyEmbeddingHelper(self.embedding,\n",
    "                                                                      start_tokens,\n",
    "                                                                      end_token)\n",
    "\n",
    "                    my_decoder = tf.contrib.seq2seq.BasicDecoder(cell,\n",
    "                                                                 helper,\n",
    "                                                                 decoder_initial_state,\n",
    "                                                                 output_layer=self.output_layer)\n",
    "                if self.inference_targets:\n",
    "                    maximum_iterations = self.maximum_iterations\n",
    "                else:\n",
    "                    maximum_iterations = None\n",
    "\n",
    "                # Dynamic decoding\n",
    "                outputs, final_context_state, _ = tf.contrib.seq2seq.dynamic_decode(\n",
    "                    my_decoder,\n",
    "                    maximum_iterations=maximum_iterations,\n",
    "                    output_time_major=False,\n",
    "                    impute_finished=False,\n",
    "                    swap_memory=False,\n",
    "                    scope=decoder_scope)\n",
    "\n",
    "                if self.beam_width > 0:\n",
    "                    logits = tf.no_op()\n",
    "                    sample_id = outputs.predicted_ids\n",
    "                else:\n",
    "                    logits = outputs.rnn_output\n",
    "                    sample_id = outputs.sample_id\n",
    "\n",
    "        return logits, sample_id, final_context_state\n",
    "\n",
    "    def build_decoder_cell(self, encoder_outputs, encoder_state,\n",
    "                           sequence_lengths_1):\n",
    "        \"\"\"Builds the attention decoder cell. If mode is inference performs tiling\n",
    "           Passes last encoder state.\n",
    "        \"\"\"\n",
    "\n",
    "        memory = encoder_outputs\n",
    "\n",
    "        if self.mode == 'INFER' and self.beam_width > 0:\n",
    "            memory = tf.contrib.seq2seq.tile_batch(memory,\n",
    "                                                   multiplier=self.beam_width)\n",
    "            encoder_state = tf.contrib.seq2seq.tile_batch(encoder_state,\n",
    "                                                          multiplier=self.beam_width)\n",
    "            sequence_lengths_1 = tf.contrib.seq2seq.tile_batch(sequence_lengths_1,\n",
    "                                                               multiplier=self.beam_width)\n",
    "            batch_size = self.batch_size * self.beam_width\n",
    "\n",
    "        else:\n",
    "            batch_size = self.batch_size\n",
    "\n",
    "        # MY APPROACH\n",
    "        if self.num_layers_decoder is not None:\n",
    "            lstm_cell = tf.nn.rnn_cell.MultiRNNCell(\n",
    "                [self.make_rnn_cell(self.rnn_size_decoder, self.keep_probability) for _ in\n",
    "                 range(self.num_layers_decoder)])\n",
    "\n",
    "        else:\n",
    "            lstm_cell = self.make_rnn_cell(self.rnn_size_decoder, self.keep_probability)\n",
    "\n",
    "        # attention cell\n",
    "        cell = self.make_attention_cell(lstm_cell,\n",
    "                                        self.rnn_size_decoder,\n",
    "                                        memory,\n",
    "                                        sequence_lengths_1)\n",
    "\n",
    "        decoder_initial_state = cell.zero_state(batch_size, tf.float32).clone(cell_state=encoder_state)\n",
    "\n",
    "        return cell, decoder_initial_state\n",
    "\n",
    "\n",
    "    def compute_loss(self, logits):\n",
    "        \"\"\"Compute the loss during optimization.\"\"\"\n",
    "        target_output = self.ids_2\n",
    "        max_time = self.maximum_iterations\n",
    "\n",
    "        target_weights = tf.sequence_mask(self.sequence_lengths_2,\n",
    "                                          max_time,\n",
    "                                          dtype=tf.float32,\n",
    "                                          name='mask')\n",
    "\n",
    "        loss = tf.contrib.seq2seq.sequence_loss(logits=logits,\n",
    "                                                targets=target_output,\n",
    "                                                weights=target_weights,\n",
    "                                                average_across_timesteps=True,\n",
    "                                                average_across_batch=True, )\n",
    "        return loss\n",
    "\n",
    "\n",
    "    def train(self,\n",
    "              inputs,\n",
    "              targets,\n",
    "              restore_path=None,\n",
    "              validation_inputs=None,\n",
    "              validation_targets=None):\n",
    "        \"\"\"Performs the training process. Runs training step in every epoch.\n",
    "           Shuffles input data before every epoch.\n",
    "           Optionally: - add tensorboard summaries.\n",
    "                       - restoring previous model and retraining on top.\n",
    "                       - evaluation step.\n",
    "        \"\"\"\n",
    "        assert len(inputs) == len(targets)\n",
    "\n",
    "        if self.summary_dir is not None:\n",
    "            self.add_summary()\n",
    "\n",
    "        self.initialize_session()\n",
    "        if restore_path is not None:\n",
    "            self.restore_session(restore_path)\n",
    "\n",
    "        best_score = np.inf\n",
    "        nepoch_no_imprv = 0\n",
    "\n",
    "        inputs = np.array(inputs)\n",
    "        targets = np.array(targets)\n",
    "\n",
    "        for epoch in range(self.epochs + 1):\n",
    "            print('-------------------- Epoch {} of {} --------------------'.format(epoch,\n",
    "                                                                                    self.epochs))\n",
    "\n",
    "            # shuffle the input data before every epoch.\n",
    "            shuffle_indices = np.random.permutation(len(inputs))\n",
    "            inputs = inputs[shuffle_indices]\n",
    "            targets = targets[shuffle_indices]\n",
    "\n",
    "            # run training epoch\n",
    "            score = self.run_epoch(inputs, targets, epoch)\n",
    "\n",
    "            # evaluate model\n",
    "            if validation_inputs is not None and validation_targets is not None:\n",
    "                self.run_evaluate(validation_inputs, validation_targets, epoch)\n",
    "\n",
    "\n",
    "            #if not os.path.exists(self.save_path):\n",
    "            #        os.makedirs(self.save_path)\n",
    "            #    self.saver.save(self.sess, self.save_path)\n",
    "                \n",
    "            if score <= best_score:\n",
    "                nepoch_no_imprv = 0\n",
    "                if not os.path.exists(self.save_path):\n",
    "                    os.makedirs(self.save_path)\n",
    "                self.saver.save(self.sess, self.save_path)\n",
    "                best_score = score\n",
    "                print(\"--- new best score ---\\n\\n\")\n",
    "            else:\n",
    "                # warm up epochs for the model\n",
    "                if epoch > 10:\n",
    "                    nepoch_no_imprv += 1\n",
    "                # early stopping\n",
    "                if nepoch_no_imprv >= 5:\n",
    "                    print(\"- early stopping {} epochs without improvement\".format(nepoch_no_imprv))\n",
    "                    break\n",
    "\n",
    "    def infer(self, inputs, restore_path, targets=None):\n",
    "        \"\"\"Runs inference process. No training takes place.\n",
    "           Returns the predicted ids for every sentence.\n",
    "        \"\"\"\n",
    "        self.initialize_session()\n",
    "        self.restore_session(restore_path)\n",
    "\n",
    "        prediction_ids = []\n",
    "        if targets is not None:\n",
    "            feed, _, sequence_lengths_2 = self.get_feed_dict(inputs, trgts=targets)\n",
    "        else:\n",
    "            feed, _ = self.get_feed_dict(inputs)\n",
    "\n",
    "        infer_logits, s_ids = self.sess.run([self.infer_logits, self.sample_words], feed_dict=feed)\n",
    "        prediction_ids.append(s_ids)\n",
    "\n",
    "        # for (inps, trgts) in summarizer_model_utils.minibatches(inputs, targets, self.batch_size):\n",
    "        #     feed, _, sequence_lengths= self.get_feed_dict(inps, trgts=trgts)\n",
    "        #     infer_logits, s_ids = self.sess.run([self.infer_logits, self.sample_words], feed_dict = feed)\n",
    "        #     prediction_ids.append(s_ids)\n",
    "\n",
    "        return prediction_ids\n",
    "\n",
    "    def run_epoch(self, inputs, targets, epoch):\n",
    "        \"\"\"Runs a single epoch.\n",
    "           Returns the average loss value on the epoch.\"\"\"\n",
    "        batch_size = self.batch_size\n",
    "        nbatches = (len(inputs) + batch_size - 1) // batch_size\n",
    "        losses = []\n",
    "\n",
    "        for i, (inps, trgts) in enumerate(minibatches(inputs,\n",
    "                                                                             targets,\n",
    "                                                                             batch_size)):\n",
    "            if inps is not None and trgts is not None:\n",
    "                fd, sl, s2 = self.get_feed_dict(inps,\n",
    "                                                trgts=trgts)\n",
    "\n",
    "                if i % 10 == 0 and self.summary_dir is not None:\n",
    "                    _, train_loss, training_summ = self.sess.run([self.train_op,\n",
    "                                                                  self.train_loss,\n",
    "                                                                  self.training_summary],\n",
    "                                                                 feed_dict=fd)\n",
    "                    self.training_writer.add_summary(training_summ, epoch*nbatches + i)\n",
    "\n",
    "                else:\n",
    "                    _, train_loss = self.sess.run([self.train_op, self.train_loss],\n",
    "                                                  feed_dict=fd)\n",
    "\n",
    "                if i % 2 == 0 or i == (nbatches - 1):\n",
    "                    print('Iteration: {} of {}\\ttrain_loss: {:.4f}'.format(i, nbatches - 1, train_loss))\n",
    "                losses.append(train_loss)\n",
    "\n",
    "            else:\n",
    "                print('Minibatch empty.')\n",
    "                continue\n",
    "\n",
    "        avg_loss = self.sess.run(tf.reduce_mean(losses))\n",
    "        print('Average Score for this Epoch: {}'.format(avg_loss))\n",
    "\n",
    "        return avg_loss\n",
    "\n",
    "    def run_evaluate(self, inputs, targets, epoch):\n",
    "        \"\"\"Runs evaluation on validation inputs and targets.\n",
    "        Optionally: - writes summary to Tensorboard.\n",
    "        \"\"\"\n",
    "        if self.summary_dir is not None:\n",
    "            eval_losses = []\n",
    "            for inps, trgts in minibatches(inputs, targets, self.batch_size):\n",
    "                fd, sl, s2 = self.get_feed_dict(inps, trgts)\n",
    "                eval_loss = self.sess.run([self.eval_loss], feed_dict=fd)\n",
    "                eval_losses.append(eval_loss)\n",
    "\n",
    "            avg_eval_loss = self.sess.run(tf.reduce_mean(eval_losses))\n",
    "\n",
    "            print('Eval_loss: {}\\n'.format(avg_eval_loss))\n",
    "            eval_summ = self.sess.run([self.eval_summary], feed_dict=fd)\n",
    "           # self.eval_writer.add_summary(eval_summ, epoch)\n",
    "\n",
    "        else:\n",
    "            eval_losses = []\n",
    "            for inps, trgts in minibatches(inputs, targets, self.batch_size):\n",
    "                fd, sl, s2 = self.get_feed_dict(inps, trgts)\n",
    "                eval_loss = self.sess.run([self.eval_loss], feed_dict=fd)\n",
    "                eval_losses.append(eval_loss)\n",
    "\n",
    "            avg_eval_loss = self.sess.run(tf.reduce_mean(eval_losses))\n",
    "\n",
    "            print('Eval_loss: {}\\n'.format(avg_eval_loss))\n",
    "\n",
    "\n",
    "\n",
    "    def get_feed_dict(self, inps, trgts=None):\n",
    "        \"\"\"Creates the feed_dict that is fed into training or inference network.\n",
    "           Pads inputs and targets.\n",
    "           Returns feed_dict and sequence_length(s) depending on training mode.\n",
    "        \"\"\"\n",
    "        if self.mode != 'INFER':\n",
    "            inp_ids, sequence_lengths_1 = pad_sequences(inps,\n",
    "                                                                               self.word2ind[self.pad],\n",
    "                                                                               tail=False)\n",
    "\n",
    "            feed = {\n",
    "                self.ids_1: inp_ids,\n",
    "                self.sequence_lengths_1: sequence_lengths_1\n",
    "            }\n",
    "\n",
    "            if trgts is not None:\n",
    "                trgt_ids, sequence_lengths_2 = pad_sequences(trgts,\n",
    "                                                                                    self.word2ind[self.pad],\n",
    "                                                                                    tail=True)\n",
    "                feed[self.ids_2] = trgt_ids\n",
    "                feed[self.sequence_lengths_2] = sequence_lengths_2\n",
    "\n",
    "                return feed, sequence_lengths_1, sequence_lengths_2\n",
    "\n",
    "        else:\n",
    "\n",
    "            inp_ids, sequence_lengths_1 = pad_sequences(inps,\n",
    "                                                                               self.word2ind[self.pad],\n",
    "                                                                               tail=False)\n",
    "\n",
    "            feed = {\n",
    "                self.ids_1: inp_ids,\n",
    "                self.sequence_lengths_1: sequence_lengths_1\n",
    "            }\n",
    "\n",
    "            if trgts is not None:\n",
    "                trgt_ids, sequence_lengths_2 = pad_sequences(trgts,\n",
    "                                                                                    self.word2ind[self.pad],\n",
    "                                                                                    tail=True)\n",
    "\n",
    "                feed[self.sequence_lengths_2] = sequence_lengths_2\n",
    "\n",
    "                return feed, sequence_lengths_1, sequence_lengths_2\n",
    "            else:\n",
    "                return feed, sequence_lengths_1\n",
    "\n",
    "    def initialize_session(self):\n",
    "        self.sess = tf.Session()\n",
    "        self.sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    def restore_session(self, restore_path):\n",
    "        self.saver.restore(self.sess, restore_path)\n",
    "        print('Done.')\n",
    "\n",
    "    def add_summary(self):\n",
    "        \"\"\"Summaries for Tensorboard.\"\"\"\n",
    "        self.training_summary = tf.summary.scalar('training_loss', self.train_loss)\n",
    "        self.eval_summary = tf.summary.scalar('evaluation_loss', self.eval_loss)\n",
    "        self.training_writer = tf.summary.FileWriter(self.summary_dir,\n",
    "                                                     tf.get_default_graph())\n",
    "        self.eval_writer = tf.summary.FileWriter(self.summary_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-XBIbvs4mw0m"
   },
   "source": [
    "## The data\n",
    "\n",
    "\n",
    "The data we will be using with is a dataset from Kaggle, the Amazon Fine Food Reviews dataset.  \n",
    "It contains, as the name suggests, 570.000 reviews of fine foods from Amazon and summaries of those reviews. \n",
    "Our aim is to input a review (Text column) and automatically create a summary (Summary colum) for it.\n",
    "\n",
    "\n",
    "https://www.kaggle.com/snap/amazon-fine-food-reviews/data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8NYVncFKmw0n"
   },
   "source": [
    "### Reading and exploring"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "2OiSxpApmw0o",
    "outputId": "237f9bbb-a0f8-4411-84b5-f307f0a58fbb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(568454, 10)"
      ]
     },
     "execution_count": 7,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load csv file using pandas.\n",
    "file_path = \"drive/Colab Notebooks/Menu/Data/Reviews.csv\"\n",
    "data = pd.read_csv(file_path)\n",
    "data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 665
    },
    "colab_type": "code",
    "id": "rElWMbT2mw0t",
    "outputId": "b300df3a-8908-477a-933f-0f231dab1265"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>ProductId</th>\n",
       "      <th>UserId</th>\n",
       "      <th>ProfileName</th>\n",
       "      <th>HelpfulnessNumerator</th>\n",
       "      <th>HelpfulnessDenominator</th>\n",
       "      <th>Score</th>\n",
       "      <th>Time</th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>B001E4KFG0</td>\n",
       "      <td>A3SGXH7AUHU8GW</td>\n",
       "      <td>delmartian</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1303862400</td>\n",
       "      <td>Good Quality Dog Food</td>\n",
       "      <td>I have bought several of the Vitality canned d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>B00813GRG4</td>\n",
       "      <td>A1D87F6ZCVE5NK</td>\n",
       "      <td>dll pa</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1346976000</td>\n",
       "      <td>Not as Advertised</td>\n",
       "      <td>Product arrived labeled as Jumbo Salted Peanut...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>B000LQOCH0</td>\n",
       "      <td>ABXLMWJIXXAIN</td>\n",
       "      <td>Natalia Corres \"Natalia Corres\"</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1219017600</td>\n",
       "      <td>\"Delight\" says it all</td>\n",
       "      <td>This is a confection that has been around a fe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>B000UA0QIQ</td>\n",
       "      <td>A395BORC6FGVXV</td>\n",
       "      <td>Karl</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1307923200</td>\n",
       "      <td>Cough Medicine</td>\n",
       "      <td>If you are looking for the secret ingredient i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>B006K2ZZ7K</td>\n",
       "      <td>A1UQRSCLF8GW1T</td>\n",
       "      <td>Michael D. Bigham \"M. Wassir\"</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1350777600</td>\n",
       "      <td>Great taffy</td>\n",
       "      <td>Great taffy at a great price.  There was a wid...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id   ProductId          UserId                      ProfileName  \\\n",
       "0   1  B001E4KFG0  A3SGXH7AUHU8GW                       delmartian   \n",
       "1   2  B00813GRG4  A1D87F6ZCVE5NK                           dll pa   \n",
       "2   3  B000LQOCH0   ABXLMWJIXXAIN  Natalia Corres \"Natalia Corres\"   \n",
       "3   4  B000UA0QIQ  A395BORC6FGVXV                             Karl   \n",
       "4   5  B006K2ZZ7K  A1UQRSCLF8GW1T    Michael D. Bigham \"M. Wassir\"   \n",
       "\n",
       "   HelpfulnessNumerator  HelpfulnessDenominator  Score        Time  \\\n",
       "0                     1                       1      5  1303862400   \n",
       "1                     0                       0      1  1346976000   \n",
       "2                     1                       1      4  1219017600   \n",
       "3                     3                       3      2  1307923200   \n",
       "4                     0                       0      5  1350777600   \n",
       "\n",
       "                 Summary                                               Text  \n",
       "0  Good Quality Dog Food  I have bought several of the Vitality canned d...  \n",
       "1      Not as Advertised  Product arrived labeled as Jumbo Salted Peanut...  \n",
       "2  \"Delight\" says it all  This is a confection that has been around a fe...  \n",
       "3         Cough Medicine  If you are looking for the secret ingredient i...  \n",
       "4            Great taffy  Great taffy at a great price.  There was a wid...  "
      ]
     },
     "execution_count": 8,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we will only use the last two columns Summary (target) and Text (input).\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 217
    },
    "colab_type": "code",
    "id": "N9bHztjpmw0x",
    "outputId": "883204fb-b17b-488a-bb07-f7677b87351e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Id                         0\n",
       "ProductId                  0\n",
       "UserId                     0\n",
       "ProfileName               16\n",
       "HelpfulnessNumerator       0\n",
       "HelpfulnessDenominator     0\n",
       "Score                      0\n",
       "Time                       0\n",
       "Summary                   27\n",
       "Text                       0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check for missings --> got some in summary drop those. \n",
    "# 26 are missing, so we will drop those!\n",
    "data.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "boMCgsgTmw00"
   },
   "outputs": [],
   "source": [
    "# drop row, if values in Summary is missing. \n",
    "data.dropna(subset=['Summary'],inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 198
    },
    "colab_type": "code",
    "id": "ESv4XLgQmw03",
    "outputId": "d6edbb05-6eed-4a67-ff2d-ee4341077d33"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Summary</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Good Quality Dog Food</td>\n",
       "      <td>I have bought several of the Vitality canned d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Not as Advertised</td>\n",
       "      <td>Product arrived labeled as Jumbo Salted Peanut...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>\"Delight\" says it all</td>\n",
       "      <td>This is a confection that has been around a fe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Cough Medicine</td>\n",
       "      <td>If you are looking for the secret ingredient i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Great taffy</td>\n",
       "      <td>Great taffy at a great price.  There was a wid...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Summary                                               Text\n",
       "0  Good Quality Dog Food  I have bought several of the Vitality canned d...\n",
       "1      Not as Advertised  Product arrived labeled as Jumbo Salted Peanut...\n",
       "2  \"Delight\" says it all  This is a confection that has been around a fe...\n",
       "3         Cough Medicine  If you are looking for the secret ingredient i...\n",
       "4            Great taffy  Great taffy at a great price.  There was a wid..."
      ]
     },
     "execution_count": 11,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# only summary and text are useful for us.\n",
    "data = data[['Summary', 'Text']]\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "BjmIGbXtmw08"
   },
   "outputs": [],
   "source": [
    "# we will not use all of them, only short ones and ones of similar size. \n",
    "# choosing the ones that are of similar length makes it easier for the model to learn.\n",
    "raw_texts = []\n",
    "raw_summaries = []\n",
    "\n",
    "for text, summary in zip(data.Text, data.Summary):\n",
    "    if 100< len(text) < 150:\n",
    "        raw_texts.append(text)\n",
    "        raw_summaries.append(summary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "t5JBoyqKmw0_",
    "outputId": "c4ee6962-4f2a-4942-c4bf-2545ef4f2bd9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(78862, 78862)"
      ]
     },
     "execution_count": 13,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(raw_texts), len(raw_summaries)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 563
    },
    "colab_type": "code",
    "id": "tMsDeec4mw1F",
    "outputId": "963bfbec-fe40-437f-9ff2-483d5fc8ac95"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Text:\n",
      " Great taffy at a great price.  There was a wide assortment of yummy taffy.  Delivery was very quick.  If your a taffy lover, this is a deal.\n",
      "Summary:\n",
      " Great taffy \n",
      "\n",
      "\n",
      "Text:\n",
      " This taffy is so good.  It is very soft and chewy.  The flavors are amazing.  I would definitely recommend you buying it.  Very satisfying!!\n",
      "Summary:\n",
      " Wonderful, tasty taffy \n",
      "\n",
      "\n",
      "Text:\n",
      " Right now I'm mostly just sprouting this so my cats can eat the grass. They love it. I rotate it around with Wheatgrass and Rye too\n",
      "Summary:\n",
      " Yay Barley \n",
      "\n",
      "\n",
      "Text:\n",
      " This is a very healthy dog food. Good for their digestion. Also good for small puppies. My dog eats her required amount at every feeding.\n",
      "Summary:\n",
      " Healthy Dog Food \n",
      "\n",
      "\n",
      "Text:\n",
      " The Strawberry Twizzlers are my guilty pleasure - yummy. Six pounds will be around for a while with my son and I.\n",
      "Summary:\n",
      " Strawberry Twizzlers - Yummy \n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for t, s in zip(raw_texts[:5], raw_summaries[:5]):\n",
    "    print('Text:\\n', t)\n",
    "    print('Summary:\\n', s, '\\n\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Y-CyKX1gmw1J"
   },
   "source": [
    "### Clean and prepare the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 72
    },
    "colab_type": "code",
    "id": "a8AGl9AIESWr",
    "outputId": "4125fdb8-e274-4043-9b92-2f9e461597f3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
      "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 17,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.download('punkt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "4CLoTqyzmw1K",
    "outputId": "638fc316-7800-4d74-b3ad-6df1c3ed0ee7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing Time:  50.83246994018555\n"
     ]
    }
   ],
   "source": [
    "# the function gives us the option to keep_most of the characters inisde the texts and summaries, meaning\n",
    "# punctuation, question marks, slashes...\n",
    "# or we can set it to False, meaning we only want to keep letters and numbers like here.\n",
    "processed_texts, processed_summaries, words_counted = preprocess_texts_and_summaries(\n",
    "    raw_texts,\n",
    "    raw_summaries,\n",
    "    keep_most=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 765
    },
    "colab_type": "code",
    "id": "yquphHYJmw1R",
    "outputId": "ed97006a-34b7-457d-9356-9ec9c5486dcc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Text\n",
      ": ['great', 'taffy', 'at', 'a', 'great', 'price', 'there', 'was', 'a', 'wide', 'assortment', 'of', 'yummy', 'taffy', 'delivery', 'was', 'very', 'quick', 'if', 'your', 'a', 'taffy', 'lover', 'this', 'is', 'a', 'deal'] \n",
      "\n",
      "Summary:\n",
      " ['great', 'taffy'] \n",
      "\n",
      "\n",
      "\n",
      "Text\n",
      ": ['this', 'taffy', 'is', 'so', 'good', 'it', 'is', 'very', 'soft', 'and', 'chewy', 'the', 'flavors', 'are', 'amazing', 'i', 'would', 'definitely', 'recommend', 'you', 'buying', 'it', 'very', 'satisfying'] \n",
      "\n",
      "Summary:\n",
      " ['wonderful', 'tasty', 'taffy'] \n",
      "\n",
      "\n",
      "\n",
      "Text\n",
      ": ['right', 'now', 'i', 'm', 'mostly', 'just', 'sprouting', 'this', 'so', 'my', 'cats', 'can', 'eat', 'the', 'grass', 'they', 'love', 'it', 'i', 'rotate', 'it', 'around', 'with', 'wheatgrass', 'and', 'rye', 'too'] \n",
      "\n",
      "Summary:\n",
      " ['yay', 'barley'] \n",
      "\n",
      "\n",
      "\n",
      "Text\n",
      ": ['this', 'is', 'a', 'very', 'healthy', 'dog', 'food', 'good', 'for', 'their', 'digestion', 'also', 'good', 'for', 'small', 'puppies', 'my', 'dog', 'eats', 'her', 'required', 'amount', 'at', 'every', 'feeding'] \n",
      "\n",
      "Summary:\n",
      " ['healthy', 'dog', 'food'] \n",
      "\n",
      "\n",
      "\n",
      "Text\n",
      ": ['the', 'strawberry', 'twizzlers', 'are', 'my', 'guilty', 'pleasure', 'yummy', 'six', 'pounds', 'will', 'be', 'around', 'for', 'a', 'while', 'with', 'my', 'son', 'and', 'i'] \n",
      "\n",
      "Summary:\n",
      " ['strawberry', 'twizzlers', 'yummy'] \n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for t,s in zip(processed_texts[:5], processed_summaries[:5]):\n",
    "    print('Text\\n:', t, '\\n')\n",
    "    print('Summary:\\n', s, '\\n\\n\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "34eUnqVQmw1c"
   },
   "source": [
    "### Create lookup dicts\n",
    "\n",
    "We cannot feed our network actual words, but numbers. So we first have to create our lookup dicts, where each words gets and int value (high or low, depending on its frequency in our corpus). Those help us to later convert the texts into numbers.\n",
    "\n",
    "We also add special tokens. EndOfSentence and StartOfSentence are crucial for the Seq2Seq model we later use.\n",
    "Pad token, because all summaries and texts in a batch need to have the same length, pad token helps us do that.\n",
    "\n",
    "So we need 2 lookup dicts:\n",
    " - From word to index \n",
    " - from index to word. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "zwqbTP8jmw1d",
    "outputId": "fc930885-dbba-47a6-804a-4a89651e8db9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25067 25067 0\n"
     ]
    }
   ],
   "source": [
    "specials = [\"<EOS>\", \"<SOS>\",\"<PAD>\",\"<UNK>\"]\n",
    "word2ind, ind2word,  missing_words = create_word_inds_dicts(words_counted,\n",
    "                                                                       specials = specials)\n",
    "print(len(word2ind), len(ind2word), len(missing_words))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EVZ1Qmk9mw1j"
   },
   "source": [
    "### Pretrained embeddings\n",
    "\n",
    "Optionally we can use pretrained word embeddings. Those have proved to increase training speed and accuracy.\n",
    "Here I used two different options. Either we use glove embeddings or embeddings from tf_hub.\n",
    "The ones from tf_hub worked better, so we use those. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8Ok2WkqFA-9P"
   },
   "outputs": [],
   "source": [
    "# glove_embeddings_path = './glove.6B.300d.txt'\n",
    "# embedding_matrix_save_path = './embeddings/my_embedding_github.npy'\n",
    "# emb = summarizer_data_utils.create_and_save_embedding_matrix(word2ind,\n",
    "#                                                              glove_embeddings_path,\n",
    "#                                                              embedding_matrix_save_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "ObE6ggfAmw1o",
    "outputId": "6f04bc62-cd09-450c-fda1-b3a79318a4a4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
     ]
    }
   ],
   "source": [
    "# the embeddings from tf_hub. \n",
    "# embed = hub.Module(\"https://tfhub.dev/google/nnlm-en-dim128/1\")\n",
    "\n",
    "\n",
    "\n",
    "embed = hub.Module(\"https://tfhub.dev/google/Wiki-words-250/1\")\n",
    "\n",
    "\n",
    "    \n",
    "emb = embed([key for key in word2ind.keys()])\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    sess.run(tf.tables_initializer())\n",
    "    embedding = sess.run(emb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "ayXi9D7Umw1u",
    "outputId": "d0dde283-dafd-4142-ec02-576bcf326aeb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(25067, 250)"
      ]
     },
     "execution_count": 30,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embedding.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QoGa9EWdmw11"
   },
   "outputs": [],
   "source": [
    "np.save('drive/Colab Notebooks/Modle 3/tf_hub_embedding.npy', embedding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QV1HB3zzmw12"
   },
   "source": [
    "### Convert text and summaries\n",
    "\n",
    "As I said before we cannot feed the words directly to our network, we have to convert them to numbers first of all. This is what we do here. And we also append the SOS and EOS tokens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NjudfxFPmw13"
   },
   "outputs": [],
   "source": [
    "# converts words in texts and summaries to indices\n",
    "# it looks like we have to set eos here to False\n",
    "converted_texts, unknown_words_in_texts = convert_to_inds(processed_texts,\n",
    "                                                                                word2ind,\n",
    "                                                                                eos = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1dFsLoAqmw16"
   },
   "outputs": [],
   "source": [
    "converted_summaries, unknown_words_in_summaries = convert_to_inds(processed_summaries,\n",
    "                                                                                        word2ind,\n",
    "                                                                                        eos = True,\n",
    "                                                                                        sos = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 508
    },
    "colab_type": "code",
    "id": "ghATcyE4mw2A",
    "outputId": "f161dc62-1da2-490b-fb2c-87c111ce6ff1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[12,\n",
       " 1727,\n",
       " 47,\n",
       " 8,\n",
       " 12,\n",
       " 45,\n",
       " 130,\n",
       " 29,\n",
       " 8,\n",
       " 2728,\n",
       " 1159,\n",
       " 15,\n",
       " 106,\n",
       " 1727,\n",
       " 322,\n",
       " 29,\n",
       " 25,\n",
       " 249,\n",
       " 62,\n",
       " 101,\n",
       " 8,\n",
       " 1727,\n",
       " 662,\n",
       " 9,\n",
       " 10,\n",
       " 8,\n",
       " 193]"
      ]
     },
     "execution_count": 37,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "converted_texts[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 55
    },
    "colab_type": "code",
    "id": "pSTzMURHmw2E",
    "outputId": "0cf99ad1-c9a0-46e3-b096-ad0004307fe1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['great', 'taffy', 'at', 'a', 'great', 'price', 'there', 'was', 'a', 'wide', 'assortment', 'of', 'yummy', 'taffy', 'delivery', 'was', 'very', 'quick', 'if', 'your', 'a', 'taffy', 'lover', 'this', 'is', 'a', 'deal'] ['<SOS>', 'great', 'taffy', '<EOS>']\n"
     ]
    }
   ],
   "source": [
    "# seems to have worked well. \n",
    "print( convert_inds_to_text(converted_texts[0], ind2word),\n",
    "       convert_inds_to_text(converted_summaries[0], ind2word))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "a8b9Nd0zmw2H"
   },
   "source": [
    "## The model\n",
    "\n",
    "Now we can build and train our model. First we define the hyperparameters we want to use. Then we create our Summarizer and call the function .build_graph(), which as the name suggests, builds the computation graph. \n",
    "Then we can train the model using .train()\n",
    "\n",
    "After training we can try our model using .infer()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "L2z9xOKzmw2I"
   },
   "source": [
    "### Training\n",
    "\n",
    "We can optionally use a cyclic learning rate, which we do here. \n",
    "I trained the model for 20 epochs and the loss was low then, but we could train it longer and would probably get better results.\n",
    "\n",
    "Unfortunately I do not have the resources to find the perfect (or right) hyperparameters, but these do pretty well. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tEItjpP4mw2J"
   },
   "outputs": [],
   "source": [
    "# model hyperparametes\n",
    "num_layers_encoder = 4\n",
    "num_layers_decoder = 4\n",
    "rnn_size_encoder = 512\n",
    "rnn_size_decoder = 512\n",
    "\n",
    "batch_size = 256\n",
    "epochs = 200\n",
    "clip = 5\n",
    "keep_probability = 0.5\n",
    "learning_rate = 0.0005\n",
    "max_lr=0.005\n",
    "learning_rate_decay_steps = 700\n",
    "learning_rate_decay = 0.90\n",
    "\n",
    "\n",
    "pretrained_embeddings_path = './tf_hub_embedding.npy'\n",
    "summary_dir = os.path.join('./tensorboard', str('Nn_' + str(rnn_size_encoder) + '_Lr_' + str(learning_rate)))\n",
    "\n",
    "\n",
    "use_cyclic_lr = True\n",
    "inference_targets=True\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "u8lJ_OI5mw2Q",
    "outputId": "9e83146b-77ad-4650-9278-07f6aeb85b0c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "78862"
      ]
     },
     "execution_count": 40,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(converted_summaries)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "w_VDuiHyQK84",
    "outputId": "7ed33041-65cd-4930-8880-917da0e7a752"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "70976"
      ]
     },
     "execution_count": 41,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "round(78862*0.9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 109427
    },
    "colab_type": "code",
    "id": "E0BX6Z7Kmw2T",
    "outputId": "b9849434-2a16-403c-a592-b4ec3834d059"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded pretrained embeddings.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gradients_impl.py:112: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.\n",
      "  \"Converting sparse IndexedSlices to a dense Tensor of unknown shape. \"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Graph built.\n",
      "-------------------- Epoch 0 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 10.1294\n",
      "Iteration: 2 of 277\ttrain_loss: 10.1122\n",
      "Iteration: 4 of 277\ttrain_loss: 10.0395\n",
      "Iteration: 6 of 277\ttrain_loss: 9.6702\n",
      "Iteration: 8 of 277\ttrain_loss: 8.7834\n",
      "Iteration: 10 of 277\ttrain_loss: 7.7196\n",
      "Iteration: 12 of 277\ttrain_loss: 6.8165\n",
      "Iteration: 14 of 277\ttrain_loss: 6.0986\n",
      "Iteration: 16 of 277\ttrain_loss: 5.4761\n",
      "Iteration: 18 of 277\ttrain_loss: 5.0323\n",
      "Iteration: 20 of 277\ttrain_loss: 5.0313\n",
      "Iteration: 22 of 277\ttrain_loss: 4.8751\n",
      "Iteration: 24 of 277\ttrain_loss: 4.8908\n",
      "Iteration: 26 of 277\ttrain_loss: 4.7569\n",
      "Iteration: 28 of 277\ttrain_loss: 4.8678\n",
      "Iteration: 30 of 277\ttrain_loss: 4.9169\n",
      "Iteration: 32 of 277\ttrain_loss: 4.7730\n",
      "Iteration: 34 of 277\ttrain_loss: 4.8769\n",
      "Iteration: 36 of 277\ttrain_loss: 4.6928\n",
      "Iteration: 38 of 277\ttrain_loss: 4.7625\n",
      "Iteration: 40 of 277\ttrain_loss: 4.8740\n",
      "Iteration: 42 of 277\ttrain_loss: 4.6174\n",
      "Iteration: 44 of 277\ttrain_loss: 4.7879\n",
      "Iteration: 46 of 277\ttrain_loss: 4.6053\n",
      "Iteration: 48 of 277\ttrain_loss: 4.5124\n",
      "Iteration: 50 of 277\ttrain_loss: 4.5271\n",
      "Iteration: 52 of 277\ttrain_loss: 4.6672\n",
      "Iteration: 54 of 277\ttrain_loss: 4.6794\n",
      "Iteration: 56 of 277\ttrain_loss: 4.6001\n",
      "Iteration: 58 of 277\ttrain_loss: 4.5813\n",
      "Iteration: 60 of 277\ttrain_loss: 4.6680\n",
      "Iteration: 62 of 277\ttrain_loss: 4.5611\n",
      "Iteration: 64 of 277\ttrain_loss: 4.6239\n",
      "Iteration: 66 of 277\ttrain_loss: 4.4706\n",
      "Iteration: 68 of 277\ttrain_loss: 4.5741\n",
      "Iteration: 70 of 277\ttrain_loss: 4.7326\n",
      "Iteration: 72 of 277\ttrain_loss: 4.6874\n",
      "Iteration: 74 of 277\ttrain_loss: 4.3486\n",
      "Iteration: 76 of 277\ttrain_loss: 4.6339\n",
      "Iteration: 78 of 277\ttrain_loss: 4.4051\n",
      "Iteration: 80 of 277\ttrain_loss: 4.3773\n",
      "Iteration: 82 of 277\ttrain_loss: 4.4531\n",
      "Iteration: 84 of 277\ttrain_loss: 4.5562\n",
      "Iteration: 86 of 277\ttrain_loss: 4.2319\n",
      "Iteration: 88 of 277\ttrain_loss: 4.5234\n",
      "Iteration: 90 of 277\ttrain_loss: 4.5544\n",
      "Iteration: 92 of 277\ttrain_loss: 4.5871\n",
      "Iteration: 94 of 277\ttrain_loss: 4.4980\n",
      "Iteration: 96 of 277\ttrain_loss: 4.5153\n",
      "Iteration: 98 of 277\ttrain_loss: 4.5518\n",
      "Iteration: 100 of 277\ttrain_loss: 4.6802\n",
      "Iteration: 102 of 277\ttrain_loss: 4.3279\n",
      "Iteration: 104 of 277\ttrain_loss: 4.5358\n",
      "Iteration: 106 of 277\ttrain_loss: 4.5433\n",
      "Iteration: 108 of 277\ttrain_loss: 4.5307\n",
      "Iteration: 110 of 277\ttrain_loss: 4.5010\n",
      "Iteration: 112 of 277\ttrain_loss: 4.6247\n",
      "Iteration: 114 of 277\ttrain_loss: 4.4740\n",
      "Iteration: 116 of 277\ttrain_loss: 4.3821\n",
      "Iteration: 118 of 277\ttrain_loss: 4.3802\n",
      "Iteration: 120 of 277\ttrain_loss: 4.4634\n",
      "Iteration: 122 of 277\ttrain_loss: 4.5559\n",
      "Iteration: 124 of 277\ttrain_loss: 4.4097\n",
      "Iteration: 126 of 277\ttrain_loss: 4.5630\n",
      "Iteration: 128 of 277\ttrain_loss: 4.5904\n",
      "Iteration: 130 of 277\ttrain_loss: 4.5915\n",
      "Iteration: 132 of 277\ttrain_loss: 4.3942\n",
      "Iteration: 134 of 277\ttrain_loss: 4.5532\n",
      "Iteration: 136 of 277\ttrain_loss: 4.4211\n",
      "Iteration: 138 of 277\ttrain_loss: 4.2896\n",
      "Iteration: 140 of 277\ttrain_loss: 4.4860\n",
      "Iteration: 142 of 277\ttrain_loss: 4.4207\n",
      "Iteration: 144 of 277\ttrain_loss: 4.4981\n",
      "Iteration: 146 of 277\ttrain_loss: 4.3548\n",
      "Iteration: 148 of 277\ttrain_loss: 4.3826\n",
      "Iteration: 150 of 277\ttrain_loss: 4.6570\n",
      "Iteration: 152 of 277\ttrain_loss: 4.4149\n",
      "Iteration: 154 of 277\ttrain_loss: 4.5346\n",
      "Iteration: 156 of 277\ttrain_loss: 4.3072\n",
      "Iteration: 158 of 277\ttrain_loss: 4.5228\n",
      "Iteration: 160 of 277\ttrain_loss: 4.5016\n",
      "Iteration: 162 of 277\ttrain_loss: 4.5423\n",
      "Iteration: 164 of 277\ttrain_loss: 4.5088\n",
      "Iteration: 166 of 277\ttrain_loss: 4.3417\n",
      "Iteration: 168 of 277\ttrain_loss: 4.5430\n",
      "Iteration: 170 of 277\ttrain_loss: 4.3883\n",
      "Iteration: 172 of 277\ttrain_loss: 4.4327\n",
      "Iteration: 174 of 277\ttrain_loss: 4.4938\n",
      "Iteration: 176 of 277\ttrain_loss: 4.4863\n",
      "Iteration: 178 of 277\ttrain_loss: 4.5616\n",
      "Iteration: 180 of 277\ttrain_loss: 4.5651\n",
      "Iteration: 182 of 277\ttrain_loss: 4.4255\n",
      "Iteration: 184 of 277\ttrain_loss: 4.6360\n",
      "Iteration: 186 of 277\ttrain_loss: 4.6207\n",
      "Iteration: 188 of 277\ttrain_loss: 4.6370\n",
      "Iteration: 190 of 277\ttrain_loss: 4.6588\n",
      "Iteration: 192 of 277\ttrain_loss: 4.5406\n",
      "Iteration: 194 of 277\ttrain_loss: 4.5709\n",
      "Iteration: 196 of 277\ttrain_loss: 4.5307\n",
      "Iteration: 198 of 277\ttrain_loss: 4.6856\n",
      "Iteration: 200 of 277\ttrain_loss: 4.5495\n",
      "Iteration: 202 of 277\ttrain_loss: 4.2060\n",
      "Iteration: 204 of 277\ttrain_loss: 4.3924\n",
      "Iteration: 206 of 277\ttrain_loss: 4.3833\n",
      "Iteration: 208 of 277\ttrain_loss: 4.6164\n",
      "Iteration: 210 of 277\ttrain_loss: 4.5597\n",
      "Iteration: 212 of 277\ttrain_loss: 4.4565\n",
      "Iteration: 214 of 277\ttrain_loss: 4.5482\n",
      "Iteration: 216 of 277\ttrain_loss: 4.6769\n",
      "Iteration: 218 of 277\ttrain_loss: 4.4932\n",
      "Iteration: 220 of 277\ttrain_loss: 4.5027\n",
      "Iteration: 222 of 277\ttrain_loss: 4.4474\n",
      "Iteration: 224 of 277\ttrain_loss: 4.5244\n",
      "Iteration: 226 of 277\ttrain_loss: 4.5792\n",
      "Iteration: 228 of 277\ttrain_loss: 4.5236\n",
      "Iteration: 230 of 277\ttrain_loss: 4.3888\n",
      "Iteration: 232 of 277\ttrain_loss: 4.4599\n",
      "Iteration: 234 of 277\ttrain_loss: 4.5536\n",
      "Iteration: 236 of 277\ttrain_loss: 4.5279\n",
      "Iteration: 238 of 277\ttrain_loss: 4.4111\n",
      "Iteration: 240 of 277\ttrain_loss: 4.4092\n",
      "Iteration: 242 of 277\ttrain_loss: 4.4720\n",
      "Iteration: 244 of 277\ttrain_loss: 4.6808\n",
      "Iteration: 246 of 277\ttrain_loss: 4.5236\n",
      "Iteration: 248 of 277\ttrain_loss: 4.6490\n",
      "Iteration: 250 of 277\ttrain_loss: 4.6721\n",
      "Iteration: 252 of 277\ttrain_loss: 4.4665\n",
      "Iteration: 254 of 277\ttrain_loss: 4.4338\n",
      "Iteration: 256 of 277\ttrain_loss: 4.6503\n",
      "Iteration: 258 of 277\ttrain_loss: 4.5826\n",
      "Iteration: 260 of 277\ttrain_loss: 4.6458\n",
      "Iteration: 262 of 277\ttrain_loss: 4.4377\n",
      "Iteration: 264 of 277\ttrain_loss: 4.5248\n",
      "Iteration: 266 of 277\ttrain_loss: 4.4994\n",
      "Iteration: 268 of 277\ttrain_loss: 4.3744\n",
      "Iteration: 270 of 277\ttrain_loss: 4.4887\n",
      "Iteration: 272 of 277\ttrain_loss: 4.4124\n",
      "Iteration: 274 of 277\ttrain_loss: 4.6820\n",
      "Iteration: 276 of 277\ttrain_loss: 4.4385\n",
      "Iteration: 277 of 277\ttrain_loss: 4.4263\n",
      "Average Score for this Epoch: 4.776391983032227\n",
      "Eval_loss: 4.4675984382629395\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 1 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 4.4334\n",
      "Iteration: 2 of 277\ttrain_loss: 4.4918\n",
      "Iteration: 4 of 277\ttrain_loss: 4.3721\n",
      "Iteration: 6 of 277\ttrain_loss: 4.2754\n",
      "Iteration: 8 of 277\ttrain_loss: 4.3945\n",
      "Iteration: 10 of 277\ttrain_loss: 4.3829\n",
      "Iteration: 12 of 277\ttrain_loss: 4.5631\n",
      "Iteration: 14 of 277\ttrain_loss: 4.3302\n",
      "Iteration: 16 of 277\ttrain_loss: 4.4714\n",
      "Iteration: 18 of 277\ttrain_loss: 4.3203\n",
      "Iteration: 20 of 277\ttrain_loss: 4.5074\n",
      "Iteration: 22 of 277\ttrain_loss: 4.4898\n",
      "Iteration: 24 of 277\ttrain_loss: 4.3368\n",
      "Iteration: 26 of 277\ttrain_loss: 4.6324\n",
      "Iteration: 28 of 277\ttrain_loss: 4.4328\n",
      "Iteration: 30 of 277\ttrain_loss: 4.4659\n",
      "Iteration: 32 of 277\ttrain_loss: 4.5534\n",
      "Iteration: 34 of 277\ttrain_loss: 4.5991\n",
      "Iteration: 36 of 277\ttrain_loss: 4.5199\n",
      "Iteration: 38 of 277\ttrain_loss: 4.4753\n",
      "Iteration: 40 of 277\ttrain_loss: 4.4794\n",
      "Iteration: 42 of 277\ttrain_loss: 4.4344\n",
      "Iteration: 44 of 277\ttrain_loss: 4.3611\n",
      "Iteration: 46 of 277\ttrain_loss: 4.4029\n",
      "Iteration: 48 of 277\ttrain_loss: 4.4096\n",
      "Iteration: 50 of 277\ttrain_loss: 4.4832\n",
      "Iteration: 52 of 277\ttrain_loss: 4.4795\n",
      "Iteration: 54 of 277\ttrain_loss: 4.4884\n",
      "Iteration: 56 of 277\ttrain_loss: 4.7884\n",
      "Iteration: 58 of 277\ttrain_loss: 4.3726\n",
      "Iteration: 60 of 277\ttrain_loss: 4.5793\n",
      "Iteration: 62 of 277\ttrain_loss: 4.4068\n",
      "Iteration: 64 of 277\ttrain_loss: 4.3747\n",
      "Iteration: 66 of 277\ttrain_loss: 4.3446\n",
      "Iteration: 68 of 277\ttrain_loss: 4.4220\n",
      "Iteration: 70 of 277\ttrain_loss: 4.4368\n",
      "Iteration: 72 of 277\ttrain_loss: 4.4798\n",
      "Iteration: 74 of 277\ttrain_loss: 4.3331\n",
      "Iteration: 76 of 277\ttrain_loss: 4.3897\n",
      "Iteration: 78 of 277\ttrain_loss: 4.1808\n",
      "Iteration: 80 of 277\ttrain_loss: 4.4597\n",
      "Iteration: 82 of 277\ttrain_loss: 4.4747\n",
      "Iteration: 84 of 277\ttrain_loss: 4.4381\n",
      "Iteration: 86 of 277\ttrain_loss: 4.3567\n",
      "Iteration: 88 of 277\ttrain_loss: 4.4708\n",
      "Iteration: 90 of 277\ttrain_loss: 4.5924\n",
      "Iteration: 92 of 277\ttrain_loss: 4.5497\n",
      "Iteration: 94 of 277\ttrain_loss: 4.2990\n",
      "Iteration: 96 of 277\ttrain_loss: 4.3220\n",
      "Iteration: 98 of 277\ttrain_loss: 4.4324\n",
      "Iteration: 100 of 277\ttrain_loss: 4.4299\n",
      "Iteration: 102 of 277\ttrain_loss: 4.5341\n",
      "Iteration: 104 of 277\ttrain_loss: 4.3786\n",
      "Iteration: 106 of 277\ttrain_loss: 4.3699\n",
      "Iteration: 108 of 277\ttrain_loss: 4.4016\n",
      "Iteration: 110 of 277\ttrain_loss: 4.3481\n",
      "Iteration: 112 of 277\ttrain_loss: 4.3381\n",
      "Iteration: 114 of 277\ttrain_loss: 4.3640\n",
      "Iteration: 116 of 277\ttrain_loss: 4.3665\n",
      "Iteration: 118 of 277\ttrain_loss: 4.3725\n",
      "Iteration: 120 of 277\ttrain_loss: 4.3320\n",
      "Iteration: 122 of 277\ttrain_loss: 4.2673\n",
      "Iteration: 124 of 277\ttrain_loss: 4.2874\n",
      "Iteration: 126 of 277\ttrain_loss: 4.4349\n",
      "Iteration: 128 of 277\ttrain_loss: 4.4030\n",
      "Iteration: 130 of 277\ttrain_loss: 4.3215\n",
      "Iteration: 132 of 277\ttrain_loss: 4.4033\n",
      "Iteration: 134 of 277\ttrain_loss: 4.4712\n",
      "Iteration: 136 of 277\ttrain_loss: 4.2855\n",
      "Iteration: 138 of 277\ttrain_loss: 4.3261\n",
      "Iteration: 140 of 277\ttrain_loss: 4.3337\n",
      "Iteration: 142 of 277\ttrain_loss: 4.3667\n",
      "Iteration: 144 of 277\ttrain_loss: 4.3454\n",
      "Iteration: 146 of 277\ttrain_loss: 4.2291\n",
      "Iteration: 148 of 277\ttrain_loss: 4.3180\n",
      "Iteration: 150 of 277\ttrain_loss: 4.3236\n",
      "Iteration: 152 of 277\ttrain_loss: 4.1214\n",
      "Iteration: 154 of 277\ttrain_loss: 4.3804\n",
      "Iteration: 156 of 277\ttrain_loss: 4.2735\n",
      "Iteration: 158 of 277\ttrain_loss: 4.2613\n",
      "Iteration: 160 of 277\ttrain_loss: 4.3895\n",
      "Iteration: 162 of 277\ttrain_loss: 4.3362\n",
      "Iteration: 164 of 277\ttrain_loss: 4.4207\n",
      "Iteration: 166 of 277\ttrain_loss: 4.4711\n",
      "Iteration: 168 of 277\ttrain_loss: 4.5016\n",
      "Iteration: 170 of 277\ttrain_loss: 4.2958\n",
      "Iteration: 172 of 277\ttrain_loss: 4.1537\n",
      "Iteration: 174 of 277\ttrain_loss: 4.2276\n",
      "Iteration: 176 of 277\ttrain_loss: 4.3182\n",
      "Iteration: 178 of 277\ttrain_loss: 4.3070\n",
      "Iteration: 180 of 277\ttrain_loss: 4.2730\n",
      "Iteration: 182 of 277\ttrain_loss: 4.4107\n",
      "Iteration: 184 of 277\ttrain_loss: 4.3563\n",
      "Iteration: 186 of 277\ttrain_loss: 4.4748\n",
      "Iteration: 188 of 277\ttrain_loss: 4.2747\n",
      "Iteration: 190 of 277\ttrain_loss: 4.3227\n",
      "Iteration: 192 of 277\ttrain_loss: 4.3697\n",
      "Iteration: 194 of 277\ttrain_loss: 4.3536\n",
      "Iteration: 196 of 277\ttrain_loss: 4.2717\n",
      "Iteration: 198 of 277\ttrain_loss: 4.4223\n",
      "Iteration: 200 of 277\ttrain_loss: 4.3695\n",
      "Iteration: 202 of 277\ttrain_loss: 4.3338\n",
      "Iteration: 204 of 277\ttrain_loss: 4.1318\n",
      "Iteration: 206 of 277\ttrain_loss: 4.3303\n",
      "Iteration: 208 of 277\ttrain_loss: 4.2732\n",
      "Iteration: 210 of 277\ttrain_loss: 4.2371\n",
      "Iteration: 212 of 277\ttrain_loss: 4.4589\n",
      "Iteration: 214 of 277\ttrain_loss: 4.3761\n",
      "Iteration: 216 of 277\ttrain_loss: 4.3427\n",
      "Iteration: 218 of 277\ttrain_loss: 4.2850\n",
      "Iteration: 220 of 277\ttrain_loss: 4.2615\n",
      "Iteration: 222 of 277\ttrain_loss: 4.2833\n",
      "Iteration: 224 of 277\ttrain_loss: 4.3058\n",
      "Iteration: 226 of 277\ttrain_loss: 4.3780\n",
      "Iteration: 228 of 277\ttrain_loss: 4.4611\n",
      "Iteration: 230 of 277\ttrain_loss: 4.3563\n",
      "Iteration: 232 of 277\ttrain_loss: 4.2205\n",
      "Iteration: 234 of 277\ttrain_loss: 4.2747\n",
      "Iteration: 236 of 277\ttrain_loss: 4.2750\n",
      "Iteration: 238 of 277\ttrain_loss: 4.3948\n",
      "Iteration: 240 of 277\ttrain_loss: 4.2044\n",
      "Iteration: 242 of 277\ttrain_loss: 4.3057\n",
      "Iteration: 244 of 277\ttrain_loss: 4.3368\n",
      "Iteration: 246 of 277\ttrain_loss: 4.3895\n",
      "Iteration: 248 of 277\ttrain_loss: 4.1242\n",
      "Iteration: 250 of 277\ttrain_loss: 4.3785\n",
      "Iteration: 252 of 277\ttrain_loss: 4.2640\n",
      "Iteration: 254 of 277\ttrain_loss: 4.2645\n",
      "Iteration: 256 of 277\ttrain_loss: 4.2791\n",
      "Iteration: 258 of 277\ttrain_loss: 4.2385\n",
      "Iteration: 260 of 277\ttrain_loss: 4.3469\n",
      "Iteration: 262 of 277\ttrain_loss: 4.3048\n",
      "Iteration: 264 of 277\ttrain_loss: 4.3501\n",
      "Iteration: 266 of 277\ttrain_loss: 4.5002\n",
      "Iteration: 268 of 277\ttrain_loss: 4.3112\n",
      "Iteration: 270 of 277\ttrain_loss: 4.2186\n",
      "Iteration: 272 of 277\ttrain_loss: 4.1841\n",
      "Iteration: 274 of 277\ttrain_loss: 4.2942\n",
      "Iteration: 276 of 277\ttrain_loss: 4.4016\n",
      "Iteration: 277 of 277\ttrain_loss: 4.1722\n",
      "Average Score for this Epoch: 4.3560614585876465\n",
      "Eval_loss: 4.253091812133789\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 2 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 4.1054\n",
      "Iteration: 2 of 277\ttrain_loss: 4.2271\n",
      "Iteration: 4 of 277\ttrain_loss: 4.1976\n",
      "Iteration: 6 of 277\ttrain_loss: 4.1811\n",
      "Iteration: 8 of 277\ttrain_loss: 4.1211\n",
      "Iteration: 10 of 277\ttrain_loss: 4.2790\n",
      "Iteration: 12 of 277\ttrain_loss: 4.2799\n",
      "Iteration: 14 of 277\ttrain_loss: 4.2166\n",
      "Iteration: 16 of 277\ttrain_loss: 4.2120\n",
      "Iteration: 18 of 277\ttrain_loss: 4.1370\n",
      "Iteration: 20 of 277\ttrain_loss: 4.2429\n",
      "Iteration: 22 of 277\ttrain_loss: 4.2734\n",
      "Iteration: 24 of 277\ttrain_loss: 4.3050\n",
      "Iteration: 26 of 277\ttrain_loss: 4.2168\n",
      "Iteration: 28 of 277\ttrain_loss: 4.3452\n",
      "Iteration: 30 of 277\ttrain_loss: 4.3870\n",
      "Iteration: 32 of 277\ttrain_loss: 4.1386\n",
      "Iteration: 34 of 277\ttrain_loss: 4.0926\n",
      "Iteration: 36 of 277\ttrain_loss: 4.2618\n",
      "Iteration: 38 of 277\ttrain_loss: 4.2821\n",
      "Iteration: 40 of 277\ttrain_loss: 4.0900\n",
      "Iteration: 42 of 277\ttrain_loss: 4.0369\n",
      "Iteration: 44 of 277\ttrain_loss: 4.2266\n",
      "Iteration: 46 of 277\ttrain_loss: 3.9310\n",
      "Iteration: 48 of 277\ttrain_loss: 4.0990\n",
      "Iteration: 50 of 277\ttrain_loss: 4.2040\n",
      "Iteration: 52 of 277\ttrain_loss: 4.3053\n",
      "Iteration: 54 of 277\ttrain_loss: 4.1494\n",
      "Iteration: 56 of 277\ttrain_loss: 4.1998\n",
      "Iteration: 58 of 277\ttrain_loss: 4.1690\n",
      "Iteration: 60 of 277\ttrain_loss: 4.0993\n",
      "Iteration: 62 of 277\ttrain_loss: 4.2115\n",
      "Iteration: 64 of 277\ttrain_loss: 4.2370\n",
      "Iteration: 66 of 277\ttrain_loss: 4.3327\n",
      "Iteration: 68 of 277\ttrain_loss: 4.2343\n",
      "Iteration: 70 of 277\ttrain_loss: 4.1554\n",
      "Iteration: 72 of 277\ttrain_loss: 4.1734\n",
      "Iteration: 74 of 277\ttrain_loss: 4.0008\n",
      "Iteration: 76 of 277\ttrain_loss: 4.1355\n",
      "Iteration: 78 of 277\ttrain_loss: 4.1539\n",
      "Iteration: 80 of 277\ttrain_loss: 4.1847\n",
      "Iteration: 82 of 277\ttrain_loss: 4.2602\n",
      "Iteration: 84 of 277\ttrain_loss: 4.1545\n",
      "Iteration: 86 of 277\ttrain_loss: 4.1232\n",
      "Iteration: 88 of 277\ttrain_loss: 4.1023\n",
      "Iteration: 90 of 277\ttrain_loss: 4.1613\n",
      "Iteration: 92 of 277\ttrain_loss: 4.1015\n",
      "Iteration: 94 of 277\ttrain_loss: 4.2264\n",
      "Iteration: 96 of 277\ttrain_loss: 3.9717\n",
      "Iteration: 98 of 277\ttrain_loss: 4.2951\n",
      "Iteration: 100 of 277\ttrain_loss: 4.1342\n",
      "Iteration: 102 of 277\ttrain_loss: 4.1339\n",
      "Iteration: 104 of 277\ttrain_loss: 4.1145\n",
      "Iteration: 106 of 277\ttrain_loss: 4.3149\n",
      "Iteration: 108 of 277\ttrain_loss: 4.2589\n",
      "Iteration: 110 of 277\ttrain_loss: 4.0954\n",
      "Iteration: 112 of 277\ttrain_loss: 4.2905\n",
      "Iteration: 114 of 277\ttrain_loss: 4.2198\n",
      "Iteration: 116 of 277\ttrain_loss: 3.9942\n",
      "Iteration: 118 of 277\ttrain_loss: 4.1614\n",
      "Iteration: 120 of 277\ttrain_loss: 4.3084\n",
      "Iteration: 122 of 277\ttrain_loss: 4.1966\n",
      "Iteration: 124 of 277\ttrain_loss: 4.1055\n",
      "Iteration: 126 of 277\ttrain_loss: 4.0724\n",
      "Iteration: 128 of 277\ttrain_loss: 4.1958\n",
      "Iteration: 130 of 277\ttrain_loss: 4.1234\n",
      "Iteration: 132 of 277\ttrain_loss: 4.0882\n",
      "Iteration: 134 of 277\ttrain_loss: 4.2292\n",
      "Iteration: 136 of 277\ttrain_loss: 4.0831\n",
      "Iteration: 138 of 277\ttrain_loss: 4.0071\n",
      "Iteration: 140 of 277\ttrain_loss: 4.1162\n",
      "Iteration: 142 of 277\ttrain_loss: 3.9362\n",
      "Iteration: 144 of 277\ttrain_loss: 3.9490\n",
      "Iteration: 146 of 277\ttrain_loss: 4.0138\n",
      "Iteration: 148 of 277\ttrain_loss: 4.2088\n",
      "Iteration: 150 of 277\ttrain_loss: 3.9440\n",
      "Iteration: 152 of 277\ttrain_loss: 4.0388\n",
      "Iteration: 154 of 277\ttrain_loss: 3.9615\n",
      "Iteration: 156 of 277\ttrain_loss: 4.0013\n",
      "Iteration: 158 of 277\ttrain_loss: 4.0562\n",
      "Iteration: 160 of 277\ttrain_loss: 3.9204\n",
      "Iteration: 162 of 277\ttrain_loss: 3.7712\n",
      "Iteration: 164 of 277\ttrain_loss: 3.9665\n",
      "Iteration: 166 of 277\ttrain_loss: 3.9775\n",
      "Iteration: 168 of 277\ttrain_loss: 3.9435\n",
      "Iteration: 170 of 277\ttrain_loss: 4.1576\n",
      "Iteration: 172 of 277\ttrain_loss: 3.9733\n",
      "Iteration: 174 of 277\ttrain_loss: 3.9624\n",
      "Iteration: 176 of 277\ttrain_loss: 3.8561\n",
      "Iteration: 178 of 277\ttrain_loss: 4.0198\n",
      "Iteration: 180 of 277\ttrain_loss: 3.9278\n",
      "Iteration: 182 of 277\ttrain_loss: 3.9840\n",
      "Iteration: 184 of 277\ttrain_loss: 3.8341\n",
      "Iteration: 186 of 277\ttrain_loss: 4.0016\n",
      "Iteration: 188 of 277\ttrain_loss: 4.0572\n",
      "Iteration: 190 of 277\ttrain_loss: 3.8302\n",
      "Iteration: 192 of 277\ttrain_loss: 3.9457\n",
      "Iteration: 194 of 277\ttrain_loss: 3.9284\n",
      "Iteration: 196 of 277\ttrain_loss: 3.8987\n",
      "Iteration: 198 of 277\ttrain_loss: 3.8871\n",
      "Iteration: 200 of 277\ttrain_loss: 3.8827\n",
      "Iteration: 202 of 277\ttrain_loss: 3.9301\n",
      "Iteration: 204 of 277\ttrain_loss: 3.8662\n",
      "Iteration: 206 of 277\ttrain_loss: 3.7997\n",
      "Iteration: 208 of 277\ttrain_loss: 3.8688\n",
      "Iteration: 210 of 277\ttrain_loss: 3.8937\n",
      "Iteration: 212 of 277\ttrain_loss: 3.9201\n",
      "Iteration: 214 of 277\ttrain_loss: 3.8026\n",
      "Iteration: 216 of 277\ttrain_loss: 3.9923\n",
      "Iteration: 218 of 277\ttrain_loss: 3.8095\n",
      "Iteration: 220 of 277\ttrain_loss: 3.9645\n",
      "Iteration: 222 of 277\ttrain_loss: 3.7712\n",
      "Iteration: 224 of 277\ttrain_loss: 3.8893\n",
      "Iteration: 226 of 277\ttrain_loss: 4.0416\n",
      "Iteration: 228 of 277\ttrain_loss: 3.7711\n",
      "Iteration: 230 of 277\ttrain_loss: 3.8558\n",
      "Iteration: 232 of 277\ttrain_loss: 3.8279\n",
      "Iteration: 234 of 277\ttrain_loss: 3.9600\n",
      "Iteration: 236 of 277\ttrain_loss: 3.7658\n",
      "Iteration: 238 of 277\ttrain_loss: 3.9078\n",
      "Iteration: 240 of 277\ttrain_loss: 3.8346\n",
      "Iteration: 242 of 277\ttrain_loss: 3.7628\n",
      "Iteration: 244 of 277\ttrain_loss: 3.9772\n",
      "Iteration: 246 of 277\ttrain_loss: 3.9070\n",
      "Iteration: 248 of 277\ttrain_loss: 3.6723\n",
      "Iteration: 250 of 277\ttrain_loss: 4.0195\n",
      "Iteration: 252 of 277\ttrain_loss: 3.7241\n",
      "Iteration: 254 of 277\ttrain_loss: 3.8488\n",
      "Iteration: 256 of 277\ttrain_loss: 3.7263\n",
      "Iteration: 258 of 277\ttrain_loss: 3.7175\n",
      "Iteration: 260 of 277\ttrain_loss: 3.7550\n",
      "Iteration: 262 of 277\ttrain_loss: 3.8374\n",
      "Iteration: 264 of 277\ttrain_loss: 3.8151\n",
      "Iteration: 266 of 277\ttrain_loss: 3.7925\n",
      "Iteration: 268 of 277\ttrain_loss: 3.6143\n",
      "Iteration: 270 of 277\ttrain_loss: 3.6849\n",
      "Iteration: 272 of 277\ttrain_loss: 3.6735\n",
      "Iteration: 274 of 277\ttrain_loss: 3.7340\n",
      "Iteration: 276 of 277\ttrain_loss: 3.7497\n",
      "Iteration: 277 of 277\ttrain_loss: 3.7819\n",
      "Average Score for this Epoch: 4.038050174713135\n",
      "Eval_loss: 3.7661263942718506\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 3 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 3.6644\n",
      "Iteration: 2 of 277\ttrain_loss: 3.7225\n",
      "Iteration: 4 of 277\ttrain_loss: 3.6705\n",
      "Iteration: 6 of 277\ttrain_loss: 3.5575\n",
      "Iteration: 8 of 277\ttrain_loss: 3.6845\n",
      "Iteration: 10 of 277\ttrain_loss: 3.5903\n",
      "Iteration: 12 of 277\ttrain_loss: 3.8236\n",
      "Iteration: 14 of 277\ttrain_loss: 3.7967\n",
      "Iteration: 16 of 277\ttrain_loss: 3.7222\n",
      "Iteration: 18 of 277\ttrain_loss: 3.5387\n",
      "Iteration: 20 of 277\ttrain_loss: 3.7675\n",
      "Iteration: 22 of 277\ttrain_loss: 3.7066\n",
      "Iteration: 24 of 277\ttrain_loss: 3.5861\n",
      "Iteration: 26 of 277\ttrain_loss: 3.5047\n",
      "Iteration: 28 of 277\ttrain_loss: 3.6095\n",
      "Iteration: 30 of 277\ttrain_loss: 3.5877\n",
      "Iteration: 32 of 277\ttrain_loss: 3.7963\n",
      "Iteration: 34 of 277\ttrain_loss: 3.5951\n",
      "Iteration: 36 of 277\ttrain_loss: 3.5650\n",
      "Iteration: 38 of 277\ttrain_loss: 3.5431\n",
      "Iteration: 40 of 277\ttrain_loss: 3.7211\n",
      "Iteration: 42 of 277\ttrain_loss: 3.6071\n",
      "Iteration: 44 of 277\ttrain_loss: 3.4726\n",
      "Iteration: 46 of 277\ttrain_loss: 3.3956\n",
      "Iteration: 48 of 277\ttrain_loss: 3.5687\n",
      "Iteration: 50 of 277\ttrain_loss: 3.5796\n",
      "Iteration: 52 of 277\ttrain_loss: 3.6132\n",
      "Iteration: 54 of 277\ttrain_loss: 3.5918\n",
      "Iteration: 56 of 277\ttrain_loss: 3.5232\n",
      "Iteration: 58 of 277\ttrain_loss: 3.5059\n",
      "Iteration: 60 of 277\ttrain_loss: 3.6822\n",
      "Iteration: 62 of 277\ttrain_loss: 3.5814\n",
      "Iteration: 64 of 277\ttrain_loss: 3.6466\n",
      "Iteration: 66 of 277\ttrain_loss: 3.5452\n",
      "Iteration: 68 of 277\ttrain_loss: 3.6049\n",
      "Iteration: 70 of 277\ttrain_loss: 3.4426\n",
      "Iteration: 72 of 277\ttrain_loss: 3.6624\n",
      "Iteration: 74 of 277\ttrain_loss: 3.4554\n",
      "Iteration: 76 of 277\ttrain_loss: 3.5532\n",
      "Iteration: 78 of 277\ttrain_loss: 3.4256\n",
      "Iteration: 80 of 277\ttrain_loss: 3.6128\n",
      "Iteration: 82 of 277\ttrain_loss: 3.5004\n",
      "Iteration: 84 of 277\ttrain_loss: 3.6295\n",
      "Iteration: 86 of 277\ttrain_loss: 3.4652\n",
      "Iteration: 88 of 277\ttrain_loss: 3.5200\n",
      "Iteration: 90 of 277\ttrain_loss: 3.5924\n",
      "Iteration: 92 of 277\ttrain_loss: 3.4414\n",
      "Iteration: 94 of 277\ttrain_loss: 3.5552\n",
      "Iteration: 96 of 277\ttrain_loss: 3.5994\n",
      "Iteration: 98 of 277\ttrain_loss: 3.6377\n",
      "Iteration: 100 of 277\ttrain_loss: 3.4970\n",
      "Iteration: 102 of 277\ttrain_loss: 3.6584\n",
      "Iteration: 104 of 277\ttrain_loss: 3.6140\n",
      "Iteration: 106 of 277\ttrain_loss: 3.6282\n",
      "Iteration: 108 of 277\ttrain_loss: 3.5125\n",
      "Iteration: 110 of 277\ttrain_loss: 3.5659\n",
      "Iteration: 112 of 277\ttrain_loss: 3.6850\n",
      "Iteration: 114 of 277\ttrain_loss: 3.6488\n",
      "Iteration: 116 of 277\ttrain_loss: 3.4523\n",
      "Iteration: 118 of 277\ttrain_loss: 3.4572\n",
      "Iteration: 120 of 277\ttrain_loss: 3.4370\n",
      "Iteration: 122 of 277\ttrain_loss: 3.4067\n",
      "Iteration: 124 of 277\ttrain_loss: 3.5959\n",
      "Iteration: 126 of 277\ttrain_loss: 3.4842\n",
      "Iteration: 128 of 277\ttrain_loss: 3.5651\n",
      "Iteration: 130 of 277\ttrain_loss: 3.4755\n",
      "Iteration: 132 of 277\ttrain_loss: 3.5405\n",
      "Iteration: 134 of 277\ttrain_loss: 3.4098\n",
      "Iteration: 136 of 277\ttrain_loss: 3.4137\n",
      "Iteration: 138 of 277\ttrain_loss: 3.6043\n",
      "Iteration: 140 of 277\ttrain_loss: 3.5855\n",
      "Iteration: 142 of 277\ttrain_loss: 3.4166\n",
      "Iteration: 144 of 277\ttrain_loss: 3.5395\n",
      "Iteration: 146 of 277\ttrain_loss: 3.4974\n",
      "Iteration: 148 of 277\ttrain_loss: 3.6656\n",
      "Iteration: 150 of 277\ttrain_loss: 3.5805\n",
      "Iteration: 152 of 277\ttrain_loss: 3.5530\n",
      "Iteration: 154 of 277\ttrain_loss: 3.5326\n",
      "Iteration: 156 of 277\ttrain_loss: 3.4819\n",
      "Iteration: 158 of 277\ttrain_loss: 3.4480\n",
      "Iteration: 160 of 277\ttrain_loss: 3.5182\n",
      "Iteration: 162 of 277\ttrain_loss: 3.3364\n",
      "Iteration: 164 of 277\ttrain_loss: 3.4543\n",
      "Iteration: 166 of 277\ttrain_loss: 3.4763\n",
      "Iteration: 168 of 277\ttrain_loss: 3.4209\n",
      "Iteration: 170 of 277\ttrain_loss: 3.6297\n",
      "Iteration: 172 of 277\ttrain_loss: 3.3513\n",
      "Iteration: 174 of 277\ttrain_loss: 3.3338\n",
      "Iteration: 176 of 277\ttrain_loss: 3.5016\n",
      "Iteration: 178 of 277\ttrain_loss: 3.6350\n",
      "Iteration: 180 of 277\ttrain_loss: 3.4801\n",
      "Iteration: 182 of 277\ttrain_loss: 3.4239\n",
      "Iteration: 184 of 277\ttrain_loss: 3.4920\n",
      "Iteration: 186 of 277\ttrain_loss: 3.4474\n",
      "Iteration: 188 of 277\ttrain_loss: 3.3290\n",
      "Iteration: 190 of 277\ttrain_loss: 3.3686\n",
      "Iteration: 192 of 277\ttrain_loss: 3.3546\n",
      "Iteration: 194 of 277\ttrain_loss: 3.3836\n",
      "Iteration: 196 of 277\ttrain_loss: 3.5495\n",
      "Iteration: 198 of 277\ttrain_loss: 3.3333\n",
      "Iteration: 200 of 277\ttrain_loss: 3.3993\n",
      "Iteration: 202 of 277\ttrain_loss: 3.5481\n",
      "Iteration: 204 of 277\ttrain_loss: 3.5424\n",
      "Iteration: 206 of 277\ttrain_loss: 3.4628\n",
      "Iteration: 208 of 277\ttrain_loss: 3.4513\n",
      "Iteration: 210 of 277\ttrain_loss: 3.5022\n",
      "Iteration: 212 of 277\ttrain_loss: 3.4536\n",
      "Iteration: 214 of 277\ttrain_loss: 3.2097\n",
      "Iteration: 216 of 277\ttrain_loss: 3.2112\n",
      "Iteration: 218 of 277\ttrain_loss: 3.5136\n",
      "Iteration: 220 of 277\ttrain_loss: 3.2549\n",
      "Iteration: 222 of 277\ttrain_loss: 3.5186\n",
      "Iteration: 224 of 277\ttrain_loss: 3.3858\n",
      "Iteration: 226 of 277\ttrain_loss: 3.3725\n",
      "Iteration: 228 of 277\ttrain_loss: 3.4034\n",
      "Iteration: 230 of 277\ttrain_loss: 3.4047\n",
      "Iteration: 232 of 277\ttrain_loss: 3.6422\n",
      "Iteration: 234 of 277\ttrain_loss: 3.3803\n",
      "Iteration: 236 of 277\ttrain_loss: 3.3355\n",
      "Iteration: 238 of 277\ttrain_loss: 3.3353\n",
      "Iteration: 240 of 277\ttrain_loss: 3.6120\n",
      "Iteration: 242 of 277\ttrain_loss: 3.3612\n",
      "Iteration: 244 of 277\ttrain_loss: 3.4114\n",
      "Iteration: 246 of 277\ttrain_loss: 3.2743\n",
      "Iteration: 248 of 277\ttrain_loss: 3.3697\n",
      "Iteration: 250 of 277\ttrain_loss: 3.3278\n",
      "Iteration: 252 of 277\ttrain_loss: 3.4184\n",
      "Iteration: 254 of 277\ttrain_loss: 3.4873\n",
      "Iteration: 256 of 277\ttrain_loss: 3.4000\n",
      "Iteration: 258 of 277\ttrain_loss: 3.3782\n",
      "Iteration: 260 of 277\ttrain_loss: 3.2542\n",
      "Iteration: 262 of 277\ttrain_loss: 3.3670\n",
      "Iteration: 264 of 277\ttrain_loss: 3.4423\n",
      "Iteration: 266 of 277\ttrain_loss: 3.4698\n",
      "Iteration: 268 of 277\ttrain_loss: 3.3980\n",
      "Iteration: 270 of 277\ttrain_loss: 3.5018\n",
      "Iteration: 272 of 277\ttrain_loss: 3.5335\n",
      "Iteration: 274 of 277\ttrain_loss: 3.3161\n",
      "Iteration: 276 of 277\ttrain_loss: 3.2433\n",
      "Iteration: 277 of 277\ttrain_loss: 3.3239\n",
      "Average Score for this Epoch: 3.5129523277282715\n",
      "Eval_loss: 3.428614616394043\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 4 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 3.3126\n",
      "Iteration: 2 of 277\ttrain_loss: 3.2560\n",
      "Iteration: 4 of 277\ttrain_loss: 3.3074\n",
      "Iteration: 6 of 277\ttrain_loss: 3.0870\n",
      "Iteration: 8 of 277\ttrain_loss: 3.2061\n",
      "Iteration: 10 of 277\ttrain_loss: 3.3386\n",
      "Iteration: 12 of 277\ttrain_loss: 3.1054\n",
      "Iteration: 14 of 277\ttrain_loss: 3.2302\n",
      "Iteration: 16 of 277\ttrain_loss: 3.3882\n",
      "Iteration: 18 of 277\ttrain_loss: 3.2178\n",
      "Iteration: 20 of 277\ttrain_loss: 3.0914\n",
      "Iteration: 22 of 277\ttrain_loss: 3.1860\n",
      "Iteration: 24 of 277\ttrain_loss: 3.2187\n",
      "Iteration: 26 of 277\ttrain_loss: 3.0759\n",
      "Iteration: 28 of 277\ttrain_loss: 3.3391\n",
      "Iteration: 30 of 277\ttrain_loss: 3.3230\n",
      "Iteration: 32 of 277\ttrain_loss: 3.2872\n",
      "Iteration: 34 of 277\ttrain_loss: 3.3325\n",
      "Iteration: 36 of 277\ttrain_loss: 3.3279\n",
      "Iteration: 38 of 277\ttrain_loss: 3.3100\n",
      "Iteration: 40 of 277\ttrain_loss: 3.2889\n",
      "Iteration: 42 of 277\ttrain_loss: 3.2939\n",
      "Iteration: 44 of 277\ttrain_loss: 3.3418\n",
      "Iteration: 46 of 277\ttrain_loss: 3.1708\n",
      "Iteration: 48 of 277\ttrain_loss: 3.1033\n",
      "Iteration: 50 of 277\ttrain_loss: 3.2017\n",
      "Iteration: 52 of 277\ttrain_loss: 3.2861\n",
      "Iteration: 54 of 277\ttrain_loss: 3.0835\n",
      "Iteration: 56 of 277\ttrain_loss: 3.0105\n",
      "Iteration: 58 of 277\ttrain_loss: 3.1651\n",
      "Iteration: 60 of 277\ttrain_loss: 3.1231\n",
      "Iteration: 62 of 277\ttrain_loss: 3.0945\n",
      "Iteration: 64 of 277\ttrain_loss: 3.2424\n",
      "Iteration: 66 of 277\ttrain_loss: 3.2173\n",
      "Iteration: 68 of 277\ttrain_loss: 3.1709\n",
      "Iteration: 70 of 277\ttrain_loss: 3.1554\n",
      "Iteration: 72 of 277\ttrain_loss: 3.2166\n",
      "Iteration: 74 of 277\ttrain_loss: 3.1526\n",
      "Iteration: 76 of 277\ttrain_loss: 3.2019\n",
      "Iteration: 78 of 277\ttrain_loss: 3.3812\n",
      "Iteration: 80 of 277\ttrain_loss: 3.3595\n",
      "Iteration: 82 of 277\ttrain_loss: 3.1050\n",
      "Iteration: 84 of 277\ttrain_loss: 3.2442\n",
      "Iteration: 86 of 277\ttrain_loss: 3.0431\n",
      "Iteration: 88 of 277\ttrain_loss: 3.3096\n",
      "Iteration: 90 of 277\ttrain_loss: 3.1971\n",
      "Iteration: 92 of 277\ttrain_loss: 3.1035\n",
      "Iteration: 94 of 277\ttrain_loss: 3.1509\n",
      "Iteration: 96 of 277\ttrain_loss: 3.1319\n",
      "Iteration: 98 of 277\ttrain_loss: 3.0304\n",
      "Iteration: 100 of 277\ttrain_loss: 3.1139\n",
      "Iteration: 102 of 277\ttrain_loss: 3.2535\n",
      "Iteration: 104 of 277\ttrain_loss: 3.3512\n",
      "Iteration: 106 of 277\ttrain_loss: 3.1698\n",
      "Iteration: 108 of 277\ttrain_loss: 3.3253\n",
      "Iteration: 110 of 277\ttrain_loss: 3.1873\n",
      "Iteration: 112 of 277\ttrain_loss: 3.0380\n",
      "Iteration: 114 of 277\ttrain_loss: 3.1190\n",
      "Iteration: 116 of 277\ttrain_loss: 3.1643\n",
      "Iteration: 118 of 277\ttrain_loss: 3.0688\n",
      "Iteration: 120 of 277\ttrain_loss: 3.1270\n",
      "Iteration: 122 of 277\ttrain_loss: 3.2669\n",
      "Iteration: 124 of 277\ttrain_loss: 3.1418\n",
      "Iteration: 126 of 277\ttrain_loss: 3.0558\n",
      "Iteration: 128 of 277\ttrain_loss: 3.1995\n",
      "Iteration: 130 of 277\ttrain_loss: 3.1965\n",
      "Iteration: 132 of 277\ttrain_loss: 3.2908\n",
      "Iteration: 134 of 277\ttrain_loss: 3.1607\n",
      "Iteration: 136 of 277\ttrain_loss: 3.2638\n",
      "Iteration: 138 of 277\ttrain_loss: 3.1925\n",
      "Iteration: 140 of 277\ttrain_loss: 3.0047\n",
      "Iteration: 142 of 277\ttrain_loss: 2.9339\n",
      "Iteration: 144 of 277\ttrain_loss: 3.0066\n",
      "Iteration: 146 of 277\ttrain_loss: 3.1128\n",
      "Iteration: 148 of 277\ttrain_loss: 3.2552\n",
      "Iteration: 150 of 277\ttrain_loss: 3.1045\n",
      "Iteration: 152 of 277\ttrain_loss: 3.1841\n",
      "Iteration: 154 of 277\ttrain_loss: 3.2718\n",
      "Iteration: 156 of 277\ttrain_loss: 3.2507\n",
      "Iteration: 158 of 277\ttrain_loss: 3.2295\n",
      "Iteration: 160 of 277\ttrain_loss: 3.1085\n",
      "Iteration: 162 of 277\ttrain_loss: 3.0333\n",
      "Iteration: 164 of 277\ttrain_loss: 3.1928\n",
      "Iteration: 166 of 277\ttrain_loss: 3.1582\n",
      "Iteration: 168 of 277\ttrain_loss: 2.9883\n",
      "Iteration: 170 of 277\ttrain_loss: 2.9709\n",
      "Iteration: 172 of 277\ttrain_loss: 2.8846\n",
      "Iteration: 174 of 277\ttrain_loss: 3.2182\n",
      "Iteration: 176 of 277\ttrain_loss: 3.2286\n",
      "Iteration: 178 of 277\ttrain_loss: 3.1213\n",
      "Iteration: 180 of 277\ttrain_loss: 2.9472\n",
      "Iteration: 182 of 277\ttrain_loss: 3.0788\n",
      "Iteration: 184 of 277\ttrain_loss: 3.3356\n",
      "Iteration: 186 of 277\ttrain_loss: 3.2478\n",
      "Iteration: 188 of 277\ttrain_loss: 3.3235\n",
      "Iteration: 190 of 277\ttrain_loss: 3.0022\n",
      "Iteration: 192 of 277\ttrain_loss: 3.1676\n",
      "Iteration: 194 of 277\ttrain_loss: 3.2120\n",
      "Iteration: 196 of 277\ttrain_loss: 3.1284\n",
      "Iteration: 198 of 277\ttrain_loss: 3.0306\n",
      "Iteration: 200 of 277\ttrain_loss: 3.1876\n",
      "Iteration: 202 of 277\ttrain_loss: 3.1304\n",
      "Iteration: 204 of 277\ttrain_loss: 3.0644\n",
      "Iteration: 206 of 277\ttrain_loss: 3.1755\n",
      "Iteration: 208 of 277\ttrain_loss: 3.1690\n",
      "Iteration: 210 of 277\ttrain_loss: 3.1268\n",
      "Iteration: 212 of 277\ttrain_loss: 3.0467\n",
      "Iteration: 214 of 277\ttrain_loss: 3.0983\n",
      "Iteration: 216 of 277\ttrain_loss: 2.9959\n",
      "Iteration: 218 of 277\ttrain_loss: 3.1361\n",
      "Iteration: 220 of 277\ttrain_loss: 3.3549\n",
      "Iteration: 222 of 277\ttrain_loss: 3.0109\n",
      "Iteration: 224 of 277\ttrain_loss: 3.1524\n",
      "Iteration: 226 of 277\ttrain_loss: 3.1608\n",
      "Iteration: 228 of 277\ttrain_loss: 3.3458\n",
      "Iteration: 230 of 277\ttrain_loss: 3.1814\n",
      "Iteration: 232 of 277\ttrain_loss: 3.1079\n",
      "Iteration: 234 of 277\ttrain_loss: 2.8744\n",
      "Iteration: 236 of 277\ttrain_loss: 3.4181\n",
      "Iteration: 238 of 277\ttrain_loss: 3.0574\n",
      "Iteration: 240 of 277\ttrain_loss: 2.9187\n",
      "Iteration: 242 of 277\ttrain_loss: 3.0968\n",
      "Iteration: 244 of 277\ttrain_loss: 3.1748\n",
      "Iteration: 246 of 277\ttrain_loss: 3.0854\n",
      "Iteration: 248 of 277\ttrain_loss: 3.0484\n",
      "Iteration: 250 of 277\ttrain_loss: 3.2025\n",
      "Iteration: 252 of 277\ttrain_loss: 3.2662\n",
      "Iteration: 254 of 277\ttrain_loss: 3.2144\n",
      "Iteration: 256 of 277\ttrain_loss: 2.9271\n",
      "Iteration: 258 of 277\ttrain_loss: 3.2286\n",
      "Iteration: 260 of 277\ttrain_loss: 3.0157\n",
      "Iteration: 262 of 277\ttrain_loss: 2.9865\n",
      "Iteration: 264 of 277\ttrain_loss: 3.2040\n",
      "Iteration: 266 of 277\ttrain_loss: 3.1664\n",
      "Iteration: 268 of 277\ttrain_loss: 3.2403\n",
      "Iteration: 270 of 277\ttrain_loss: 3.2473\n",
      "Iteration: 272 of 277\ttrain_loss: 3.0152\n",
      "Iteration: 274 of 277\ttrain_loss: 3.0496\n",
      "Iteration: 276 of 277\ttrain_loss: 3.2483\n",
      "Iteration: 277 of 277\ttrain_loss: 3.1072\n",
      "Average Score for this Epoch: 3.1740963459014893\n",
      "Eval_loss: 3.259364366531372\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 5 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 3.0023\n",
      "Iteration: 2 of 277\ttrain_loss: 2.9937\n",
      "Iteration: 4 of 277\ttrain_loss: 2.8301\n",
      "Iteration: 6 of 277\ttrain_loss: 3.0148\n",
      "Iteration: 8 of 277\ttrain_loss: 2.9228\n",
      "Iteration: 10 of 277\ttrain_loss: 3.1396\n",
      "Iteration: 12 of 277\ttrain_loss: 3.1270\n",
      "Iteration: 14 of 277\ttrain_loss: 3.0657\n",
      "Iteration: 16 of 277\ttrain_loss: 3.0116\n",
      "Iteration: 18 of 277\ttrain_loss: 3.0739\n",
      "Iteration: 20 of 277\ttrain_loss: 2.9865\n",
      "Iteration: 22 of 277\ttrain_loss: 3.1109\n",
      "Iteration: 24 of 277\ttrain_loss: 3.1837\n",
      "Iteration: 26 of 277\ttrain_loss: 2.8905\n",
      "Iteration: 28 of 277\ttrain_loss: 3.1307\n",
      "Iteration: 30 of 277\ttrain_loss: 3.0226\n",
      "Iteration: 32 of 277\ttrain_loss: 2.8531\n",
      "Iteration: 34 of 277\ttrain_loss: 2.9303\n",
      "Iteration: 36 of 277\ttrain_loss: 2.9773\n",
      "Iteration: 38 of 277\ttrain_loss: 2.9357\n",
      "Iteration: 40 of 277\ttrain_loss: 3.1732\n",
      "Iteration: 42 of 277\ttrain_loss: 3.0735\n",
      "Iteration: 44 of 277\ttrain_loss: 3.1879\n",
      "Iteration: 46 of 277\ttrain_loss: 3.0848\n",
      "Iteration: 48 of 277\ttrain_loss: 2.9213\n",
      "Iteration: 50 of 277\ttrain_loss: 3.0981\n",
      "Iteration: 52 of 277\ttrain_loss: 3.0736\n",
      "Iteration: 54 of 277\ttrain_loss: 3.0221\n",
      "Iteration: 56 of 277\ttrain_loss: 2.8642\n",
      "Iteration: 58 of 277\ttrain_loss: 3.1304\n",
      "Iteration: 60 of 277\ttrain_loss: 2.9672\n",
      "Iteration: 62 of 277\ttrain_loss: 3.0926\n",
      "Iteration: 64 of 277\ttrain_loss: 2.8859\n",
      "Iteration: 66 of 277\ttrain_loss: 3.1566\n",
      "Iteration: 68 of 277\ttrain_loss: 2.8991\n",
      "Iteration: 70 of 277\ttrain_loss: 3.0361\n",
      "Iteration: 72 of 277\ttrain_loss: 3.0649\n",
      "Iteration: 74 of 277\ttrain_loss: 2.9308\n",
      "Iteration: 76 of 277\ttrain_loss: 3.0163\n",
      "Iteration: 78 of 277\ttrain_loss: 2.9689\n",
      "Iteration: 80 of 277\ttrain_loss: 2.9460\n",
      "Iteration: 82 of 277\ttrain_loss: 3.0051\n",
      "Iteration: 84 of 277\ttrain_loss: 2.8774\n",
      "Iteration: 86 of 277\ttrain_loss: 2.9346\n",
      "Iteration: 88 of 277\ttrain_loss: 2.9955\n",
      "Iteration: 90 of 277\ttrain_loss: 3.0471\n",
      "Iteration: 92 of 277\ttrain_loss: 3.1440\n",
      "Iteration: 94 of 277\ttrain_loss: 3.1290\n",
      "Iteration: 96 of 277\ttrain_loss: 3.0138\n",
      "Iteration: 98 of 277\ttrain_loss: 2.9161\n",
      "Iteration: 100 of 277\ttrain_loss: 3.0448\n",
      "Iteration: 102 of 277\ttrain_loss: 2.9666\n",
      "Iteration: 104 of 277\ttrain_loss: 3.0354\n",
      "Iteration: 106 of 277\ttrain_loss: 2.9290\n",
      "Iteration: 108 of 277\ttrain_loss: 3.0100\n",
      "Iteration: 110 of 277\ttrain_loss: 3.0353\n",
      "Iteration: 112 of 277\ttrain_loss: 2.9067\n",
      "Iteration: 114 of 277\ttrain_loss: 3.0631\n",
      "Iteration: 116 of 277\ttrain_loss: 2.9823\n",
      "Iteration: 118 of 277\ttrain_loss: 3.1746\n",
      "Iteration: 120 of 277\ttrain_loss: 3.0539\n",
      "Iteration: 122 of 277\ttrain_loss: 3.1120\n",
      "Iteration: 124 of 277\ttrain_loss: 3.0756\n",
      "Iteration: 126 of 277\ttrain_loss: 3.0393\n",
      "Iteration: 128 of 277\ttrain_loss: 3.0090\n",
      "Iteration: 130 of 277\ttrain_loss: 2.8965\n",
      "Iteration: 132 of 277\ttrain_loss: 3.0514\n",
      "Iteration: 134 of 277\ttrain_loss: 2.9289\n",
      "Iteration: 136 of 277\ttrain_loss: 3.0796\n",
      "Iteration: 138 of 277\ttrain_loss: 2.9930\n",
      "Iteration: 140 of 277\ttrain_loss: 2.9962\n",
      "Iteration: 142 of 277\ttrain_loss: 2.8808\n",
      "Iteration: 144 of 277\ttrain_loss: 2.9816\n",
      "Iteration: 146 of 277\ttrain_loss: 2.8093\n",
      "Iteration: 148 of 277\ttrain_loss: 2.9209\n",
      "Iteration: 150 of 277\ttrain_loss: 2.9756\n",
      "Iteration: 152 of 277\ttrain_loss: 3.1391\n",
      "Iteration: 154 of 277\ttrain_loss: 3.0742\n",
      "Iteration: 156 of 277\ttrain_loss: 2.9935\n",
      "Iteration: 158 of 277\ttrain_loss: 3.0339\n",
      "Iteration: 160 of 277\ttrain_loss: 3.0415\n",
      "Iteration: 162 of 277\ttrain_loss: 3.2046\n",
      "Iteration: 164 of 277\ttrain_loss: 3.1372\n",
      "Iteration: 166 of 277\ttrain_loss: 2.9426\n",
      "Iteration: 168 of 277\ttrain_loss: 2.9963\n",
      "Iteration: 170 of 277\ttrain_loss: 3.1321\n",
      "Iteration: 172 of 277\ttrain_loss: 2.9731\n",
      "Iteration: 174 of 277\ttrain_loss: 2.9190\n",
      "Iteration: 176 of 277\ttrain_loss: 2.9726\n",
      "Iteration: 178 of 277\ttrain_loss: 3.0847\n",
      "Iteration: 180 of 277\ttrain_loss: 3.0130\n",
      "Iteration: 182 of 277\ttrain_loss: 3.0168\n",
      "Iteration: 184 of 277\ttrain_loss: 2.9845\n",
      "Iteration: 186 of 277\ttrain_loss: 2.8915\n",
      "Iteration: 188 of 277\ttrain_loss: 3.1966\n",
      "Iteration: 190 of 277\ttrain_loss: 2.9497\n",
      "Iteration: 192 of 277\ttrain_loss: 3.1174\n",
      "Iteration: 194 of 277\ttrain_loss: 2.9253\n",
      "Iteration: 196 of 277\ttrain_loss: 2.9109\n",
      "Iteration: 198 of 277\ttrain_loss: 2.8844\n",
      "Iteration: 200 of 277\ttrain_loss: 2.9203\n",
      "Iteration: 202 of 277\ttrain_loss: 2.9705\n",
      "Iteration: 204 of 277\ttrain_loss: 3.0485\n",
      "Iteration: 206 of 277\ttrain_loss: 2.9824\n",
      "Iteration: 208 of 277\ttrain_loss: 3.0328\n",
      "Iteration: 210 of 277\ttrain_loss: 2.9376\n",
      "Iteration: 212 of 277\ttrain_loss: 3.0646\n",
      "Iteration: 214 of 277\ttrain_loss: 3.0145\n",
      "Iteration: 216 of 277\ttrain_loss: 2.8905\n",
      "Iteration: 218 of 277\ttrain_loss: 2.9740\n",
      "Iteration: 220 of 277\ttrain_loss: 3.0358\n",
      "Iteration: 222 of 277\ttrain_loss: 3.0166\n",
      "Iteration: 224 of 277\ttrain_loss: 2.9976\n",
      "Iteration: 226 of 277\ttrain_loss: 2.9742\n",
      "Iteration: 228 of 277\ttrain_loss: 2.8170\n",
      "Iteration: 230 of 277\ttrain_loss: 3.0409\n",
      "Iteration: 232 of 277\ttrain_loss: 2.8802\n",
      "Iteration: 234 of 277\ttrain_loss: 2.5999\n",
      "Iteration: 236 of 277\ttrain_loss: 2.9033\n",
      "Iteration: 238 of 277\ttrain_loss: 3.0488\n",
      "Iteration: 240 of 277\ttrain_loss: 2.9652\n",
      "Iteration: 242 of 277\ttrain_loss: 2.9716\n",
      "Iteration: 244 of 277\ttrain_loss: 3.0279\n",
      "Iteration: 246 of 277\ttrain_loss: 3.0114\n",
      "Iteration: 248 of 277\ttrain_loss: 2.8232\n",
      "Iteration: 250 of 277\ttrain_loss: 3.0680\n",
      "Iteration: 252 of 277\ttrain_loss: 3.0790\n",
      "Iteration: 254 of 277\ttrain_loss: 3.0404\n",
      "Iteration: 256 of 277\ttrain_loss: 2.9707\n",
      "Iteration: 258 of 277\ttrain_loss: 2.9517\n",
      "Iteration: 260 of 277\ttrain_loss: 2.9167\n",
      "Iteration: 262 of 277\ttrain_loss: 3.0126\n",
      "Iteration: 264 of 277\ttrain_loss: 2.8995\n",
      "Iteration: 266 of 277\ttrain_loss: 2.8453\n",
      "Iteration: 268 of 277\ttrain_loss: 2.9862\n",
      "Iteration: 270 of 277\ttrain_loss: 2.7522\n",
      "Iteration: 272 of 277\ttrain_loss: 2.8929\n",
      "Iteration: 274 of 277\ttrain_loss: 2.9230\n",
      "Iteration: 276 of 277\ttrain_loss: 2.9426\n",
      "Iteration: 277 of 277\ttrain_loss: 2.9593\n",
      "Average Score for this Epoch: 3.002899169921875\n",
      "Eval_loss: 3.1710667610168457\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 6 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 2.7831\n",
      "Iteration: 2 of 277\ttrain_loss: 2.7683\n",
      "Iteration: 4 of 277\ttrain_loss: 2.8069\n",
      "Iteration: 6 of 277\ttrain_loss: 3.0466\n",
      "Iteration: 8 of 277\ttrain_loss: 3.1023\n",
      "Iteration: 10 of 277\ttrain_loss: 2.8627\n",
      "Iteration: 12 of 277\ttrain_loss: 2.9328\n",
      "Iteration: 14 of 277\ttrain_loss: 2.8582\n",
      "Iteration: 16 of 277\ttrain_loss: 2.8501\n",
      "Iteration: 18 of 277\ttrain_loss: 2.8365\n",
      "Iteration: 20 of 277\ttrain_loss: 2.8434\n",
      "Iteration: 22 of 277\ttrain_loss: 2.8262\n",
      "Iteration: 24 of 277\ttrain_loss: 2.9412\n",
      "Iteration: 26 of 277\ttrain_loss: 2.8328\n",
      "Iteration: 28 of 277\ttrain_loss: 2.9045\n",
      "Iteration: 30 of 277\ttrain_loss: 2.8513\n",
      "Iteration: 32 of 277\ttrain_loss: 2.8160\n",
      "Iteration: 34 of 277\ttrain_loss: 2.9073\n",
      "Iteration: 36 of 277\ttrain_loss: 2.8263\n",
      "Iteration: 38 of 277\ttrain_loss: 2.9034\n",
      "Iteration: 40 of 277\ttrain_loss: 2.7249\n",
      "Iteration: 42 of 277\ttrain_loss: 2.9657\n",
      "Iteration: 44 of 277\ttrain_loss: 2.9748\n",
      "Iteration: 46 of 277\ttrain_loss: 3.0504\n",
      "Iteration: 48 of 277\ttrain_loss: 2.7836\n",
      "Iteration: 50 of 277\ttrain_loss: 2.8044\n",
      "Iteration: 52 of 277\ttrain_loss: 3.0107\n",
      "Iteration: 54 of 277\ttrain_loss: 2.9565\n",
      "Iteration: 56 of 277\ttrain_loss: 2.7076\n",
      "Iteration: 58 of 277\ttrain_loss: 2.8551\n",
      "Iteration: 60 of 277\ttrain_loss: 2.7971\n",
      "Iteration: 62 of 277\ttrain_loss: 2.7452\n",
      "Iteration: 64 of 277\ttrain_loss: 2.8001\n",
      "Iteration: 66 of 277\ttrain_loss: 2.8180\n",
      "Iteration: 68 of 277\ttrain_loss: 2.9156\n",
      "Iteration: 70 of 277\ttrain_loss: 2.9855\n",
      "Iteration: 72 of 277\ttrain_loss: 2.8111\n",
      "Iteration: 74 of 277\ttrain_loss: 2.9232\n",
      "Iteration: 76 of 277\ttrain_loss: 2.9171\n",
      "Iteration: 78 of 277\ttrain_loss: 2.8657\n",
      "Iteration: 80 of 277\ttrain_loss: 2.6898\n",
      "Iteration: 82 of 277\ttrain_loss: 2.9983\n",
      "Iteration: 84 of 277\ttrain_loss: 2.8125\n",
      "Iteration: 86 of 277\ttrain_loss: 2.7596\n",
      "Iteration: 88 of 277\ttrain_loss: 3.0358\n",
      "Iteration: 90 of 277\ttrain_loss: 2.8940\n",
      "Iteration: 92 of 277\ttrain_loss: 2.7245\n",
      "Iteration: 94 of 277\ttrain_loss: 2.9377\n",
      "Iteration: 96 of 277\ttrain_loss: 2.8760\n",
      "Iteration: 98 of 277\ttrain_loss: 2.8395\n",
      "Iteration: 100 of 277\ttrain_loss: 2.8824\n",
      "Iteration: 102 of 277\ttrain_loss: 2.9669\n",
      "Iteration: 104 of 277\ttrain_loss: 2.8563\n",
      "Iteration: 106 of 277\ttrain_loss: 2.6998\n",
      "Iteration: 108 of 277\ttrain_loss: 2.7260\n",
      "Iteration: 110 of 277\ttrain_loss: 2.9153\n",
      "Iteration: 112 of 277\ttrain_loss: 2.7992\n",
      "Iteration: 114 of 277\ttrain_loss: 2.8705\n",
      "Iteration: 116 of 277\ttrain_loss: 2.9070\n",
      "Iteration: 118 of 277\ttrain_loss: 2.9091\n",
      "Iteration: 120 of 277\ttrain_loss: 2.7905\n",
      "Iteration: 122 of 277\ttrain_loss: 2.8103\n",
      "Iteration: 124 of 277\ttrain_loss: 2.9174\n",
      "Iteration: 126 of 277\ttrain_loss: 2.8315\n",
      "Iteration: 128 of 277\ttrain_loss: 2.8404\n",
      "Iteration: 130 of 277\ttrain_loss: 3.1024\n",
      "Iteration: 132 of 277\ttrain_loss: 2.6845\n",
      "Iteration: 134 of 277\ttrain_loss: 2.6773\n",
      "Iteration: 136 of 277\ttrain_loss: 2.9285\n",
      "Iteration: 138 of 277\ttrain_loss: 2.9131\n",
      "Iteration: 140 of 277\ttrain_loss: 3.0620\n",
      "Iteration: 142 of 277\ttrain_loss: 3.0151\n",
      "Iteration: 144 of 277\ttrain_loss: 2.6742\n",
      "Iteration: 146 of 277\ttrain_loss: 3.0710\n",
      "Iteration: 148 of 277\ttrain_loss: 2.7687\n",
      "Iteration: 150 of 277\ttrain_loss: 2.7646\n",
      "Iteration: 152 of 277\ttrain_loss: 2.9123\n",
      "Iteration: 154 of 277\ttrain_loss: 2.8758\n",
      "Iteration: 156 of 277\ttrain_loss: 3.0215\n",
      "Iteration: 158 of 277\ttrain_loss: 3.0771\n",
      "Iteration: 160 of 277\ttrain_loss: 2.9603\n",
      "Iteration: 162 of 277\ttrain_loss: 2.9475\n",
      "Iteration: 164 of 277\ttrain_loss: 3.0295\n",
      "Iteration: 166 of 277\ttrain_loss: 2.7473\n",
      "Iteration: 168 of 277\ttrain_loss: 2.9314\n",
      "Iteration: 170 of 277\ttrain_loss: 3.1421\n",
      "Iteration: 172 of 277\ttrain_loss: 3.0338\n",
      "Iteration: 174 of 277\ttrain_loss: 2.9542\n",
      "Iteration: 176 of 277\ttrain_loss: 2.8853\n",
      "Iteration: 178 of 277\ttrain_loss: 2.8567\n",
      "Iteration: 180 of 277\ttrain_loss: 2.8659\n",
      "Iteration: 182 of 277\ttrain_loss: 2.8632\n",
      "Iteration: 184 of 277\ttrain_loss: 3.0875\n",
      "Iteration: 186 of 277\ttrain_loss: 2.9922\n",
      "Iteration: 188 of 277\ttrain_loss: 3.0281\n",
      "Iteration: 190 of 277\ttrain_loss: 2.8655\n",
      "Iteration: 192 of 277\ttrain_loss: 2.8229\n",
      "Iteration: 194 of 277\ttrain_loss: 2.8473\n",
      "Iteration: 196 of 277\ttrain_loss: 2.9094\n",
      "Iteration: 198 of 277\ttrain_loss: 2.7885\n",
      "Iteration: 200 of 277\ttrain_loss: 2.8030\n",
      "Iteration: 202 of 277\ttrain_loss: 2.8370\n",
      "Iteration: 204 of 277\ttrain_loss: 2.7959\n",
      "Iteration: 206 of 277\ttrain_loss: 2.8563\n",
      "Iteration: 208 of 277\ttrain_loss: 2.6836\n",
      "Iteration: 210 of 277\ttrain_loss: 2.9001\n",
      "Iteration: 212 of 277\ttrain_loss: 2.8115\n",
      "Iteration: 214 of 277\ttrain_loss: 2.8264\n",
      "Iteration: 216 of 277\ttrain_loss: 2.9900\n",
      "Iteration: 218 of 277\ttrain_loss: 2.9054\n",
      "Iteration: 220 of 277\ttrain_loss: 2.6893\n",
      "Iteration: 222 of 277\ttrain_loss: 2.7871\n",
      "Iteration: 224 of 277\ttrain_loss: 2.7721\n",
      "Iteration: 226 of 277\ttrain_loss: 2.8038\n",
      "Iteration: 228 of 277\ttrain_loss: 2.8425\n",
      "Iteration: 230 of 277\ttrain_loss: 2.7871\n",
      "Iteration: 232 of 277\ttrain_loss: 2.6213\n",
      "Iteration: 234 of 277\ttrain_loss: 2.8452\n",
      "Iteration: 236 of 277\ttrain_loss: 2.8478\n",
      "Iteration: 238 of 277\ttrain_loss: 2.7658\n",
      "Iteration: 240 of 277\ttrain_loss: 2.7711\n",
      "Iteration: 242 of 277\ttrain_loss: 2.8461\n",
      "Iteration: 244 of 277\ttrain_loss: 2.9465\n",
      "Iteration: 246 of 277\ttrain_loss: 2.7239\n",
      "Iteration: 248 of 277\ttrain_loss: 2.7510\n",
      "Iteration: 250 of 277\ttrain_loss: 2.9609\n",
      "Iteration: 252 of 277\ttrain_loss: 2.8409\n",
      "Iteration: 254 of 277\ttrain_loss: 2.8635\n",
      "Iteration: 256 of 277\ttrain_loss: 2.7961\n",
      "Iteration: 258 of 277\ttrain_loss: 2.8223\n",
      "Iteration: 260 of 277\ttrain_loss: 2.8907\n",
      "Iteration: 262 of 277\ttrain_loss: 2.7697\n",
      "Iteration: 264 of 277\ttrain_loss: 2.6290\n",
      "Iteration: 266 of 277\ttrain_loss: 2.7919\n",
      "Iteration: 268 of 277\ttrain_loss: 2.8394\n",
      "Iteration: 270 of 277\ttrain_loss: 2.8233\n",
      "Iteration: 272 of 277\ttrain_loss: 2.7584\n",
      "Iteration: 274 of 277\ttrain_loss: 2.6139\n",
      "Iteration: 276 of 277\ttrain_loss: 2.9835\n",
      "Iteration: 277 of 277\ttrain_loss: 2.6578\n",
      "Average Score for this Epoch: 2.865295648574829\n",
      "Eval_loss: 3.0324904918670654\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 7 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 2.6028\n",
      "Iteration: 2 of 277\ttrain_loss: 2.4981\n",
      "Iteration: 4 of 277\ttrain_loss: 2.6069\n",
      "Iteration: 6 of 277\ttrain_loss: 2.6714\n",
      "Iteration: 8 of 277\ttrain_loss: 2.5083\n",
      "Iteration: 10 of 277\ttrain_loss: 2.8307\n",
      "Iteration: 12 of 277\ttrain_loss: 2.8396\n",
      "Iteration: 14 of 277\ttrain_loss: 2.6951\n",
      "Iteration: 16 of 277\ttrain_loss: 2.7282\n",
      "Iteration: 18 of 277\ttrain_loss: 2.6120\n",
      "Iteration: 20 of 277\ttrain_loss: 2.5401\n",
      "Iteration: 22 of 277\ttrain_loss: 2.7857\n",
      "Iteration: 24 of 277\ttrain_loss: 2.6553\n",
      "Iteration: 26 of 277\ttrain_loss: 2.5633\n",
      "Iteration: 28 of 277\ttrain_loss: 3.0763\n",
      "Iteration: 30 of 277\ttrain_loss: 2.7105\n",
      "Iteration: 32 of 277\ttrain_loss: 2.8083\n",
      "Iteration: 34 of 277\ttrain_loss: 2.6899\n",
      "Iteration: 36 of 277\ttrain_loss: 2.8930\n",
      "Iteration: 38 of 277\ttrain_loss: 2.6862\n",
      "Iteration: 40 of 277\ttrain_loss: 2.8015\n",
      "Iteration: 42 of 277\ttrain_loss: 2.8482\n",
      "Iteration: 44 of 277\ttrain_loss: 2.4349\n",
      "Iteration: 46 of 277\ttrain_loss: 2.6472\n",
      "Iteration: 48 of 277\ttrain_loss: 2.6083\n",
      "Iteration: 50 of 277\ttrain_loss: 2.6120\n",
      "Iteration: 52 of 277\ttrain_loss: 2.7421\n",
      "Iteration: 54 of 277\ttrain_loss: 2.7880\n",
      "Iteration: 56 of 277\ttrain_loss: 2.4720\n",
      "Iteration: 58 of 277\ttrain_loss: 2.4282\n",
      "Iteration: 60 of 277\ttrain_loss: 2.6848\n",
      "Iteration: 62 of 277\ttrain_loss: 2.5877\n",
      "Iteration: 64 of 277\ttrain_loss: 2.6937\n",
      "Iteration: 66 of 277\ttrain_loss: 2.8003\n",
      "Iteration: 68 of 277\ttrain_loss: 2.7146\n",
      "Iteration: 70 of 277\ttrain_loss: 2.6155\n",
      "Iteration: 72 of 277\ttrain_loss: 2.6619\n",
      "Iteration: 74 of 277\ttrain_loss: 2.5279\n",
      "Iteration: 76 of 277\ttrain_loss: 2.7211\n",
      "Iteration: 78 of 277\ttrain_loss: 2.7123\n",
      "Iteration: 80 of 277\ttrain_loss: 2.6996\n",
      "Iteration: 82 of 277\ttrain_loss: 2.8209\n",
      "Iteration: 84 of 277\ttrain_loss: 2.6904\n",
      "Iteration: 86 of 277\ttrain_loss: 2.7720\n",
      "Iteration: 88 of 277\ttrain_loss: 2.6204\n",
      "Iteration: 90 of 277\ttrain_loss: 2.5292\n",
      "Iteration: 92 of 277\ttrain_loss: 2.6549\n",
      "Iteration: 94 of 277\ttrain_loss: 2.6749\n",
      "Iteration: 96 of 277\ttrain_loss: 2.6770\n",
      "Iteration: 98 of 277\ttrain_loss: 2.7173\n",
      "Iteration: 100 of 277\ttrain_loss: 2.5798\n",
      "Iteration: 102 of 277\ttrain_loss: 2.5711\n",
      "Iteration: 104 of 277\ttrain_loss: 2.5151\n",
      "Iteration: 106 of 277\ttrain_loss: 2.9164\n",
      "Iteration: 108 of 277\ttrain_loss: 2.7281\n",
      "Iteration: 110 of 277\ttrain_loss: 2.7837\n",
      "Iteration: 112 of 277\ttrain_loss: 2.7711\n",
      "Iteration: 114 of 277\ttrain_loss: 2.7610\n",
      "Iteration: 116 of 277\ttrain_loss: 2.7081\n",
      "Iteration: 118 of 277\ttrain_loss: 2.6148\n",
      "Iteration: 120 of 277\ttrain_loss: 2.8551\n",
      "Iteration: 122 of 277\ttrain_loss: 2.6758\n",
      "Iteration: 124 of 277\ttrain_loss: 2.8437\n",
      "Iteration: 126 of 277\ttrain_loss: 2.6364\n",
      "Iteration: 128 of 277\ttrain_loss: 2.8340\n",
      "Iteration: 130 of 277\ttrain_loss: 2.8235\n",
      "Iteration: 132 of 277\ttrain_loss: 2.4768\n",
      "Iteration: 134 of 277\ttrain_loss: 2.7556\n",
      "Iteration: 136 of 277\ttrain_loss: 2.5462\n",
      "Iteration: 138 of 277\ttrain_loss: 2.5753\n",
      "Iteration: 140 of 277\ttrain_loss: 2.4998\n",
      "Iteration: 142 of 277\ttrain_loss: 2.6569\n",
      "Iteration: 144 of 277\ttrain_loss: 2.4609\n",
      "Iteration: 146 of 277\ttrain_loss: 2.5577\n",
      "Iteration: 148 of 277\ttrain_loss: 2.6706\n",
      "Iteration: 150 of 277\ttrain_loss: 2.7093\n",
      "Iteration: 152 of 277\ttrain_loss: 2.8652\n",
      "Iteration: 154 of 277\ttrain_loss: 2.7585\n",
      "Iteration: 156 of 277\ttrain_loss: 2.6650\n",
      "Iteration: 158 of 277\ttrain_loss: 2.6866\n",
      "Iteration: 160 of 277\ttrain_loss: 2.5972\n",
      "Iteration: 162 of 277\ttrain_loss: 2.7453\n",
      "Iteration: 164 of 277\ttrain_loss: 2.6369\n",
      "Iteration: 166 of 277\ttrain_loss: 2.7441\n",
      "Iteration: 168 of 277\ttrain_loss: 2.5335\n",
      "Iteration: 170 of 277\ttrain_loss: 2.7436\n",
      "Iteration: 172 of 277\ttrain_loss: 2.5134\n",
      "Iteration: 174 of 277\ttrain_loss: 2.7755\n",
      "Iteration: 176 of 277\ttrain_loss: 2.6572\n",
      "Iteration: 178 of 277\ttrain_loss: 2.6574\n",
      "Iteration: 180 of 277\ttrain_loss: 2.5684\n",
      "Iteration: 182 of 277\ttrain_loss: 2.6605\n",
      "Iteration: 184 of 277\ttrain_loss: 2.6378\n",
      "Iteration: 186 of 277\ttrain_loss: 2.9160\n",
      "Iteration: 188 of 277\ttrain_loss: 2.6336\n",
      "Iteration: 190 of 277\ttrain_loss: 2.6112\n",
      "Iteration: 192 of 277\ttrain_loss: 2.8502\n",
      "Iteration: 194 of 277\ttrain_loss: 2.6278\n",
      "Iteration: 196 of 277\ttrain_loss: 2.6058\n",
      "Iteration: 198 of 277\ttrain_loss: 2.7405\n",
      "Iteration: 200 of 277\ttrain_loss: 2.6108\n",
      "Iteration: 202 of 277\ttrain_loss: 2.4876\n",
      "Iteration: 204 of 277\ttrain_loss: 2.5248\n",
      "Iteration: 206 of 277\ttrain_loss: 2.6138\n",
      "Iteration: 208 of 277\ttrain_loss: 2.6467\n",
      "Iteration: 210 of 277\ttrain_loss: 2.7277\n",
      "Iteration: 212 of 277\ttrain_loss: 2.6630\n",
      "Iteration: 214 of 277\ttrain_loss: 2.6016\n",
      "Iteration: 216 of 277\ttrain_loss: 2.4028\n",
      "Iteration: 218 of 277\ttrain_loss: 2.7026\n",
      "Iteration: 220 of 277\ttrain_loss: 2.6816\n",
      "Iteration: 222 of 277\ttrain_loss: 2.5394\n",
      "Iteration: 224 of 277\ttrain_loss: 2.4693\n",
      "Iteration: 226 of 277\ttrain_loss: 2.5079\n",
      "Iteration: 228 of 277\ttrain_loss: 2.6227\n",
      "Iteration: 230 of 277\ttrain_loss: 2.3661\n",
      "Iteration: 232 of 277\ttrain_loss: 2.6478\n",
      "Iteration: 234 of 277\ttrain_loss: 2.8656\n",
      "Iteration: 236 of 277\ttrain_loss: 2.6964\n",
      "Iteration: 238 of 277\ttrain_loss: 2.7074\n",
      "Iteration: 240 of 277\ttrain_loss: 2.5572\n",
      "Iteration: 242 of 277\ttrain_loss: 2.5355\n",
      "Iteration: 244 of 277\ttrain_loss: 2.7234\n",
      "Iteration: 246 of 277\ttrain_loss: 2.7229\n",
      "Iteration: 248 of 277\ttrain_loss: 2.6578\n",
      "Iteration: 250 of 277\ttrain_loss: 2.5996\n",
      "Iteration: 252 of 277\ttrain_loss: 2.5362\n",
      "Iteration: 254 of 277\ttrain_loss: 2.8275\n",
      "Iteration: 256 of 277\ttrain_loss: 2.6679\n",
      "Iteration: 258 of 277\ttrain_loss: 2.7475\n",
      "Iteration: 260 of 277\ttrain_loss: 2.5434\n",
      "Iteration: 262 of 277\ttrain_loss: 2.8261\n",
      "Iteration: 264 of 277\ttrain_loss: 2.4994\n",
      "Iteration: 266 of 277\ttrain_loss: 2.7658\n",
      "Iteration: 268 of 277\ttrain_loss: 2.5913\n",
      "Iteration: 270 of 277\ttrain_loss: 3.0462\n",
      "Iteration: 272 of 277\ttrain_loss: 2.7456\n",
      "Iteration: 274 of 277\ttrain_loss: 2.6349\n",
      "Iteration: 276 of 277\ttrain_loss: 2.5004\n",
      "Iteration: 277 of 277\ttrain_loss: 2.4539\n",
      "Average Score for this Epoch: 2.6670026779174805\n",
      "Eval_loss: 2.8827285766601562\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 8 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 2.3868\n",
      "Iteration: 2 of 277\ttrain_loss: 2.3045\n",
      "Iteration: 4 of 277\ttrain_loss: 2.3806\n",
      "Iteration: 6 of 277\ttrain_loss: 2.4455\n",
      "Iteration: 8 of 277\ttrain_loss: 2.5139\n",
      "Iteration: 10 of 277\ttrain_loss: 2.4719\n",
      "Iteration: 12 of 277\ttrain_loss: 2.4038\n",
      "Iteration: 14 of 277\ttrain_loss: 2.4915\n",
      "Iteration: 16 of 277\ttrain_loss: 2.4608\n",
      "Iteration: 18 of 277\ttrain_loss: 2.5285\n",
      "Iteration: 20 of 277\ttrain_loss: 2.3794\n",
      "Iteration: 22 of 277\ttrain_loss: 2.5088\n",
      "Iteration: 24 of 277\ttrain_loss: 2.4215\n",
      "Iteration: 26 of 277\ttrain_loss: 2.4442\n",
      "Iteration: 28 of 277\ttrain_loss: 2.4962\n",
      "Iteration: 30 of 277\ttrain_loss: 2.4720\n",
      "Iteration: 32 of 277\ttrain_loss: 2.5064\n",
      "Iteration: 34 of 277\ttrain_loss: 2.2588\n",
      "Iteration: 36 of 277\ttrain_loss: 2.4760\n",
      "Iteration: 38 of 277\ttrain_loss: 2.3606\n",
      "Iteration: 40 of 277\ttrain_loss: 2.3146\n",
      "Iteration: 42 of 277\ttrain_loss: 2.3313\n",
      "Iteration: 44 of 277\ttrain_loss: 2.4409\n",
      "Iteration: 46 of 277\ttrain_loss: 2.4760\n",
      "Iteration: 48 of 277\ttrain_loss: 2.2660\n",
      "Iteration: 50 of 277\ttrain_loss: 2.5772\n",
      "Iteration: 52 of 277\ttrain_loss: 2.4475\n",
      "Iteration: 54 of 277\ttrain_loss: 2.3451\n",
      "Iteration: 56 of 277\ttrain_loss: 2.4454\n",
      "Iteration: 58 of 277\ttrain_loss: 2.3956\n",
      "Iteration: 60 of 277\ttrain_loss: 2.2939\n",
      "Iteration: 62 of 277\ttrain_loss: 2.3284\n",
      "Iteration: 64 of 277\ttrain_loss: 2.6993\n",
      "Iteration: 66 of 277\ttrain_loss: 2.5483\n",
      "Iteration: 68 of 277\ttrain_loss: 2.3076\n",
      "Iteration: 70 of 277\ttrain_loss: 2.4592\n",
      "Iteration: 72 of 277\ttrain_loss: 2.5340\n",
      "Iteration: 74 of 277\ttrain_loss: 2.3797\n",
      "Iteration: 76 of 277\ttrain_loss: 2.3237\n",
      "Iteration: 78 of 277\ttrain_loss: 2.4033\n",
      "Iteration: 80 of 277\ttrain_loss: 2.1470\n",
      "Iteration: 82 of 277\ttrain_loss: 2.4547\n",
      "Iteration: 84 of 277\ttrain_loss: 2.3914\n",
      "Iteration: 86 of 277\ttrain_loss: 2.4541\n",
      "Iteration: 88 of 277\ttrain_loss: 2.3868\n",
      "Iteration: 90 of 277\ttrain_loss: 2.4978\n",
      "Iteration: 92 of 277\ttrain_loss: 2.3284\n",
      "Iteration: 94 of 277\ttrain_loss: 2.5841\n",
      "Iteration: 96 of 277\ttrain_loss: 2.3449\n",
      "Iteration: 98 of 277\ttrain_loss: 2.4368\n",
      "Iteration: 100 of 277\ttrain_loss: 2.3686\n",
      "Iteration: 102 of 277\ttrain_loss: 2.2418\n",
      "Iteration: 104 of 277\ttrain_loss: 2.4580\n",
      "Iteration: 106 of 277\ttrain_loss: 2.4717\n",
      "Iteration: 108 of 277\ttrain_loss: 2.4540\n",
      "Iteration: 110 of 277\ttrain_loss: 2.2452\n",
      "Iteration: 112 of 277\ttrain_loss: 2.4015\n",
      "Iteration: 114 of 277\ttrain_loss: 2.5601\n",
      "Iteration: 116 of 277\ttrain_loss: 2.3585\n",
      "Iteration: 118 of 277\ttrain_loss: 2.1528\n",
      "Iteration: 120 of 277\ttrain_loss: 2.4367\n",
      "Iteration: 122 of 277\ttrain_loss: 2.3300\n",
      "Iteration: 124 of 277\ttrain_loss: 2.3883\n",
      "Iteration: 126 of 277\ttrain_loss: 2.4607\n",
      "Iteration: 128 of 277\ttrain_loss: 2.4244\n",
      "Iteration: 130 of 277\ttrain_loss: 2.4511\n",
      "Iteration: 132 of 277\ttrain_loss: 2.3336\n",
      "Iteration: 134 of 277\ttrain_loss: 2.3561\n",
      "Iteration: 136 of 277\ttrain_loss: 2.4354\n",
      "Iteration: 138 of 277\ttrain_loss: 2.4425\n",
      "Iteration: 140 of 277\ttrain_loss: 2.2463\n",
      "Iteration: 142 of 277\ttrain_loss: 2.2604\n",
      "Iteration: 144 of 277\ttrain_loss: 2.4135\n",
      "Iteration: 146 of 277\ttrain_loss: 2.7579\n",
      "Iteration: 148 of 277\ttrain_loss: 2.4249\n",
      "Iteration: 150 of 277\ttrain_loss: 2.3119\n",
      "Iteration: 152 of 277\ttrain_loss: 2.5314\n",
      "Iteration: 154 of 277\ttrain_loss: 2.4750\n",
      "Iteration: 156 of 277\ttrain_loss: 2.4007\n",
      "Iteration: 158 of 277\ttrain_loss: 2.2159\n",
      "Iteration: 160 of 277\ttrain_loss: 2.3991\n",
      "Iteration: 162 of 277\ttrain_loss: 2.3251\n",
      "Iteration: 164 of 277\ttrain_loss: 2.4978\n",
      "Iteration: 166 of 277\ttrain_loss: 2.2935\n",
      "Iteration: 168 of 277\ttrain_loss: 2.4343\n",
      "Iteration: 170 of 277\ttrain_loss: 2.4899\n",
      "Iteration: 172 of 277\ttrain_loss: 2.3541\n",
      "Iteration: 174 of 277\ttrain_loss: 2.5206\n",
      "Iteration: 176 of 277\ttrain_loss: 2.4299\n",
      "Iteration: 178 of 277\ttrain_loss: 2.5813\n",
      "Iteration: 180 of 277\ttrain_loss: 2.3158\n",
      "Iteration: 182 of 277\ttrain_loss: 2.4965\n",
      "Iteration: 184 of 277\ttrain_loss: 2.4305\n",
      "Iteration: 186 of 277\ttrain_loss: 2.2785\n",
      "Iteration: 188 of 277\ttrain_loss: 2.4970\n",
      "Iteration: 190 of 277\ttrain_loss: 2.4676\n",
      "Iteration: 192 of 277\ttrain_loss: 2.4973\n",
      "Iteration: 194 of 277\ttrain_loss: 2.4197\n",
      "Iteration: 196 of 277\ttrain_loss: 2.4603\n",
      "Iteration: 198 of 277\ttrain_loss: 2.3895\n",
      "Iteration: 200 of 277\ttrain_loss: 2.2908\n",
      "Iteration: 202 of 277\ttrain_loss: 2.3949\n",
      "Iteration: 204 of 277\ttrain_loss: 2.4595\n",
      "Iteration: 206 of 277\ttrain_loss: 2.3291\n",
      "Iteration: 208 of 277\ttrain_loss: 2.5383\n",
      "Iteration: 210 of 277\ttrain_loss: 2.3193\n",
      "Iteration: 212 of 277\ttrain_loss: 2.2968\n",
      "Iteration: 214 of 277\ttrain_loss: 2.2891\n",
      "Iteration: 216 of 277\ttrain_loss: 2.3794\n",
      "Iteration: 218 of 277\ttrain_loss: 2.4533\n",
      "Iteration: 220 of 277\ttrain_loss: 2.1727\n",
      "Iteration: 222 of 277\ttrain_loss: 2.4988\n",
      "Iteration: 224 of 277\ttrain_loss: 2.1795\n",
      "Iteration: 226 of 277\ttrain_loss: 2.3079\n",
      "Iteration: 228 of 277\ttrain_loss: 2.1983\n",
      "Iteration: 230 of 277\ttrain_loss: 2.1951\n",
      "Iteration: 232 of 277\ttrain_loss: 2.3464\n",
      "Iteration: 234 of 277\ttrain_loss: 2.3599\n",
      "Iteration: 236 of 277\ttrain_loss: 2.3275\n",
      "Iteration: 238 of 277\ttrain_loss: 2.2424\n",
      "Iteration: 240 of 277\ttrain_loss: 2.6808\n",
      "Iteration: 242 of 277\ttrain_loss: 2.1270\n",
      "Iteration: 244 of 277\ttrain_loss: 2.2703\n",
      "Iteration: 246 of 277\ttrain_loss: 2.3376\n",
      "Iteration: 248 of 277\ttrain_loss: 2.4800\n",
      "Iteration: 250 of 277\ttrain_loss: 2.3797\n",
      "Iteration: 252 of 277\ttrain_loss: 2.1141\n",
      "Iteration: 254 of 277\ttrain_loss: 2.4053\n",
      "Iteration: 256 of 277\ttrain_loss: 2.3227\n",
      "Iteration: 258 of 277\ttrain_loss: 2.1988\n",
      "Iteration: 260 of 277\ttrain_loss: 2.3843\n",
      "Iteration: 262 of 277\ttrain_loss: 2.2158\n",
      "Iteration: 264 of 277\ttrain_loss: 2.4062\n",
      "Iteration: 266 of 277\ttrain_loss: 2.1971\n",
      "Iteration: 268 of 277\ttrain_loss: 2.2592\n",
      "Iteration: 270 of 277\ttrain_loss: 2.3909\n",
      "Iteration: 272 of 277\ttrain_loss: 2.2761\n",
      "Iteration: 274 of 277\ttrain_loss: 2.5075\n",
      "Iteration: 276 of 277\ttrain_loss: 2.4289\n",
      "Iteration: 277 of 277\ttrain_loss: 2.1529\n",
      "Average Score for this Epoch: 2.390622615814209\n",
      "Eval_loss: 2.7529361248016357\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 9 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 2.2600\n",
      "Iteration: 2 of 277\ttrain_loss: 1.9881\n",
      "Iteration: 4 of 277\ttrain_loss: 2.1041\n",
      "Iteration: 6 of 277\ttrain_loss: 2.0037\n",
      "Iteration: 8 of 277\ttrain_loss: 2.2411\n",
      "Iteration: 10 of 277\ttrain_loss: 2.3109\n",
      "Iteration: 12 of 277\ttrain_loss: 2.1038\n",
      "Iteration: 14 of 277\ttrain_loss: 2.0809\n",
      "Iteration: 16 of 277\ttrain_loss: 2.1702\n",
      "Iteration: 18 of 277\ttrain_loss: 2.3058\n",
      "Iteration: 20 of 277\ttrain_loss: 2.2009\n",
      "Iteration: 22 of 277\ttrain_loss: 2.1526\n",
      "Iteration: 24 of 277\ttrain_loss: 2.2548\n",
      "Iteration: 26 of 277\ttrain_loss: 2.1326\n",
      "Iteration: 28 of 277\ttrain_loss: 2.1900\n",
      "Iteration: 30 of 277\ttrain_loss: 2.1202\n",
      "Iteration: 32 of 277\ttrain_loss: 2.1295\n",
      "Iteration: 34 of 277\ttrain_loss: 2.2352\n",
      "Iteration: 36 of 277\ttrain_loss: 2.0813\n",
      "Iteration: 38 of 277\ttrain_loss: 2.3294\n",
      "Iteration: 40 of 277\ttrain_loss: 2.1914\n",
      "Iteration: 42 of 277\ttrain_loss: 2.1445\n",
      "Iteration: 44 of 277\ttrain_loss: 2.2789\n",
      "Iteration: 46 of 277\ttrain_loss: 2.1155\n",
      "Iteration: 48 of 277\ttrain_loss: 2.2823\n",
      "Iteration: 50 of 277\ttrain_loss: 2.1320\n",
      "Iteration: 52 of 277\ttrain_loss: 2.2184\n",
      "Iteration: 54 of 277\ttrain_loss: 2.0171\n",
      "Iteration: 56 of 277\ttrain_loss: 2.0370\n",
      "Iteration: 58 of 277\ttrain_loss: 2.1532\n",
      "Iteration: 60 of 277\ttrain_loss: 2.1421\n",
      "Iteration: 62 of 277\ttrain_loss: 2.1663\n",
      "Iteration: 64 of 277\ttrain_loss: 2.0602\n",
      "Iteration: 66 of 277\ttrain_loss: 2.0793\n",
      "Iteration: 68 of 277\ttrain_loss: 2.1192\n",
      "Iteration: 70 of 277\ttrain_loss: 2.2337\n",
      "Iteration: 72 of 277\ttrain_loss: 2.0305\n",
      "Iteration: 74 of 277\ttrain_loss: 2.0023\n",
      "Iteration: 76 of 277\ttrain_loss: 2.2099\n",
      "Iteration: 78 of 277\ttrain_loss: 2.2321\n",
      "Iteration: 80 of 277\ttrain_loss: 2.2261\n",
      "Iteration: 82 of 277\ttrain_loss: 2.1693\n",
      "Iteration: 84 of 277\ttrain_loss: 2.0365\n",
      "Iteration: 86 of 277\ttrain_loss: 2.1766\n",
      "Iteration: 88 of 277\ttrain_loss: 2.3034\n",
      "Iteration: 90 of 277\ttrain_loss: 2.0340\n",
      "Iteration: 92 of 277\ttrain_loss: 2.1788\n",
      "Iteration: 94 of 277\ttrain_loss: 2.1388\n",
      "Iteration: 96 of 277\ttrain_loss: 2.1499\n",
      "Iteration: 98 of 277\ttrain_loss: 2.1968\n",
      "Iteration: 100 of 277\ttrain_loss: 2.3558\n",
      "Iteration: 102 of 277\ttrain_loss: 2.0972\n",
      "Iteration: 104 of 277\ttrain_loss: 2.3094\n",
      "Iteration: 106 of 277\ttrain_loss: 2.1620\n",
      "Iteration: 108 of 277\ttrain_loss: 2.1873\n",
      "Iteration: 110 of 277\ttrain_loss: 2.0484\n",
      "Iteration: 112 of 277\ttrain_loss: 2.3480\n",
      "Iteration: 114 of 277\ttrain_loss: 2.1544\n",
      "Iteration: 116 of 277\ttrain_loss: 2.1390\n",
      "Iteration: 118 of 277\ttrain_loss: 2.2233\n",
      "Iteration: 120 of 277\ttrain_loss: 1.8645\n",
      "Iteration: 122 of 277\ttrain_loss: 2.1054\n",
      "Iteration: 124 of 277\ttrain_loss: 2.5345\n",
      "Iteration: 126 of 277\ttrain_loss: 2.2434\n",
      "Iteration: 128 of 277\ttrain_loss: 2.3961\n",
      "Iteration: 130 of 277\ttrain_loss: 1.9140\n",
      "Iteration: 132 of 277\ttrain_loss: 2.1865\n",
      "Iteration: 134 of 277\ttrain_loss: 2.2263\n",
      "Iteration: 136 of 277\ttrain_loss: 2.1645\n",
      "Iteration: 138 of 277\ttrain_loss: 2.2979\n",
      "Iteration: 140 of 277\ttrain_loss: 2.1743\n",
      "Iteration: 142 of 277\ttrain_loss: 2.0725\n",
      "Iteration: 144 of 277\ttrain_loss: 1.9616\n",
      "Iteration: 146 of 277\ttrain_loss: 2.2372\n",
      "Iteration: 148 of 277\ttrain_loss: 2.2643\n",
      "Iteration: 150 of 277\ttrain_loss: 2.0085\n",
      "Iteration: 152 of 277\ttrain_loss: 2.2026\n",
      "Iteration: 154 of 277\ttrain_loss: 2.1729\n",
      "Iteration: 156 of 277\ttrain_loss: 1.8679\n",
      "Iteration: 158 of 277\ttrain_loss: 2.0934\n",
      "Iteration: 160 of 277\ttrain_loss: 2.2080\n",
      "Iteration: 162 of 277\ttrain_loss: 2.0416\n",
      "Iteration: 164 of 277\ttrain_loss: 2.2207\n",
      "Iteration: 166 of 277\ttrain_loss: 2.2810\n",
      "Iteration: 168 of 277\ttrain_loss: 2.0155\n",
      "Iteration: 170 of 277\ttrain_loss: 2.1655\n",
      "Iteration: 172 of 277\ttrain_loss: 2.0750\n",
      "Iteration: 174 of 277\ttrain_loss: 2.1922\n",
      "Iteration: 176 of 277\ttrain_loss: 1.9598\n",
      "Iteration: 178 of 277\ttrain_loss: 2.1947\n",
      "Iteration: 180 of 277\ttrain_loss: 2.2560\n",
      "Iteration: 182 of 277\ttrain_loss: 2.0779\n",
      "Iteration: 184 of 277\ttrain_loss: 2.1828\n",
      "Iteration: 186 of 277\ttrain_loss: 2.2303\n",
      "Iteration: 188 of 277\ttrain_loss: 2.1844\n",
      "Iteration: 190 of 277\ttrain_loss: 2.1005\n",
      "Iteration: 192 of 277\ttrain_loss: 2.2883\n",
      "Iteration: 194 of 277\ttrain_loss: 2.1510\n",
      "Iteration: 196 of 277\ttrain_loss: 2.3214\n",
      "Iteration: 198 of 277\ttrain_loss: 2.1154\n",
      "Iteration: 200 of 277\ttrain_loss: 2.1378\n",
      "Iteration: 202 of 277\ttrain_loss: 2.2225\n",
      "Iteration: 204 of 277\ttrain_loss: 2.0537\n",
      "Iteration: 206 of 277\ttrain_loss: 2.2625\n",
      "Iteration: 208 of 277\ttrain_loss: 1.9949\n",
      "Iteration: 210 of 277\ttrain_loss: 2.0158\n",
      "Iteration: 212 of 277\ttrain_loss: 2.1513\n",
      "Iteration: 214 of 277\ttrain_loss: 2.1940\n",
      "Iteration: 216 of 277\ttrain_loss: 2.1851\n",
      "Iteration: 218 of 277\ttrain_loss: 2.0751\n",
      "Iteration: 220 of 277\ttrain_loss: 2.0458\n",
      "Iteration: 222 of 277\ttrain_loss: 2.0972\n",
      "Iteration: 224 of 277\ttrain_loss: 2.0003\n",
      "Iteration: 226 of 277\ttrain_loss: 2.2387\n",
      "Iteration: 228 of 277\ttrain_loss: 2.1602\n",
      "Iteration: 230 of 277\ttrain_loss: 2.1758\n",
      "Iteration: 232 of 277\ttrain_loss: 2.1899\n",
      "Iteration: 234 of 277\ttrain_loss: 2.2121\n",
      "Iteration: 236 of 277\ttrain_loss: 2.2015\n",
      "Iteration: 238 of 277\ttrain_loss: 2.0573\n",
      "Iteration: 240 of 277\ttrain_loss: 2.1855\n",
      "Iteration: 242 of 277\ttrain_loss: 2.1821\n",
      "Iteration: 244 of 277\ttrain_loss: 2.0437\n",
      "Iteration: 246 of 277\ttrain_loss: 2.1688\n",
      "Iteration: 248 of 277\ttrain_loss: 2.0207\n",
      "Iteration: 250 of 277\ttrain_loss: 1.9244\n",
      "Iteration: 252 of 277\ttrain_loss: 2.2923\n",
      "Iteration: 254 of 277\ttrain_loss: 2.1160\n",
      "Iteration: 256 of 277\ttrain_loss: 2.0609\n",
      "Iteration: 258 of 277\ttrain_loss: 2.3800\n",
      "Iteration: 260 of 277\ttrain_loss: 2.0878\n",
      "Iteration: 262 of 277\ttrain_loss: 2.0675\n",
      "Iteration: 264 of 277\ttrain_loss: 2.2181\n",
      "Iteration: 266 of 277\ttrain_loss: 2.0871\n",
      "Iteration: 268 of 277\ttrain_loss: 2.0421\n",
      "Iteration: 270 of 277\ttrain_loss: 2.0023\n",
      "Iteration: 272 of 277\ttrain_loss: 1.9536\n",
      "Iteration: 274 of 277\ttrain_loss: 2.0396\n",
      "Iteration: 276 of 277\ttrain_loss: 2.1551\n",
      "Iteration: 277 of 277\ttrain_loss: 1.9779\n",
      "Average Score for this Epoch: 2.151766538619995\n",
      "Eval_loss: 2.6771318912506104\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 10 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 2.0965\n",
      "Iteration: 2 of 277\ttrain_loss: 2.1560\n",
      "Iteration: 4 of 277\ttrain_loss: 2.0037\n",
      "Iteration: 6 of 277\ttrain_loss: 2.0203\n",
      "Iteration: 8 of 277\ttrain_loss: 2.3402\n",
      "Iteration: 10 of 277\ttrain_loss: 2.1316\n",
      "Iteration: 12 of 277\ttrain_loss: 2.0198\n",
      "Iteration: 14 of 277\ttrain_loss: 2.0004\n",
      "Iteration: 16 of 277\ttrain_loss: 1.9471\n",
      "Iteration: 18 of 277\ttrain_loss: 2.2775\n",
      "Iteration: 20 of 277\ttrain_loss: 1.9784\n",
      "Iteration: 22 of 277\ttrain_loss: 1.9310\n",
      "Iteration: 24 of 277\ttrain_loss: 1.9002\n",
      "Iteration: 26 of 277\ttrain_loss: 2.1466\n",
      "Iteration: 28 of 277\ttrain_loss: 2.0238\n",
      "Iteration: 30 of 277\ttrain_loss: 2.0754\n",
      "Iteration: 32 of 277\ttrain_loss: 2.0707\n",
      "Iteration: 34 of 277\ttrain_loss: 2.0624\n",
      "Iteration: 36 of 277\ttrain_loss: 1.9294\n",
      "Iteration: 38 of 277\ttrain_loss: 1.9734\n",
      "Iteration: 40 of 277\ttrain_loss: 2.1000\n",
      "Iteration: 42 of 277\ttrain_loss: 2.1072\n",
      "Iteration: 44 of 277\ttrain_loss: 1.9741\n",
      "Iteration: 46 of 277\ttrain_loss: 2.1193\n",
      "Iteration: 48 of 277\ttrain_loss: 2.0847\n",
      "Iteration: 50 of 277\ttrain_loss: 1.8248\n",
      "Iteration: 52 of 277\ttrain_loss: 2.0230\n",
      "Iteration: 54 of 277\ttrain_loss: 1.9856\n",
      "Iteration: 56 of 277\ttrain_loss: 2.0358\n",
      "Iteration: 58 of 277\ttrain_loss: 1.9730\n",
      "Iteration: 60 of 277\ttrain_loss: 1.8820\n",
      "Iteration: 62 of 277\ttrain_loss: 2.0575\n",
      "Iteration: 64 of 277\ttrain_loss: 1.9203\n",
      "Iteration: 66 of 277\ttrain_loss: 1.9250\n",
      "Iteration: 68 of 277\ttrain_loss: 2.1914\n",
      "Iteration: 70 of 277\ttrain_loss: 1.9947\n",
      "Iteration: 72 of 277\ttrain_loss: 1.9779\n",
      "Iteration: 74 of 277\ttrain_loss: 1.8864\n",
      "Iteration: 76 of 277\ttrain_loss: 2.0042\n",
      "Iteration: 78 of 277\ttrain_loss: 2.0042\n",
      "Iteration: 80 of 277\ttrain_loss: 2.1492\n",
      "Iteration: 82 of 277\ttrain_loss: 2.1351\n",
      "Iteration: 84 of 277\ttrain_loss: 2.1150\n",
      "Iteration: 86 of 277\ttrain_loss: 2.0708\n",
      "Iteration: 88 of 277\ttrain_loss: 1.9966\n",
      "Iteration: 90 of 277\ttrain_loss: 2.1844\n",
      "Iteration: 92 of 277\ttrain_loss: 2.0822\n",
      "Iteration: 94 of 277\ttrain_loss: 2.1062\n",
      "Iteration: 96 of 277\ttrain_loss: 1.8995\n",
      "Iteration: 98 of 277\ttrain_loss: 1.9548\n",
      "Iteration: 100 of 277\ttrain_loss: 2.0399\n",
      "Iteration: 102 of 277\ttrain_loss: 1.9283\n",
      "Iteration: 104 of 277\ttrain_loss: 1.9813\n",
      "Iteration: 106 of 277\ttrain_loss: 2.0549\n",
      "Iteration: 108 of 277\ttrain_loss: 1.9197\n",
      "Iteration: 110 of 277\ttrain_loss: 1.9854\n",
      "Iteration: 112 of 277\ttrain_loss: 2.1351\n",
      "Iteration: 114 of 277\ttrain_loss: 2.0376\n",
      "Iteration: 116 of 277\ttrain_loss: 1.9560\n",
      "Iteration: 118 of 277\ttrain_loss: 1.9574\n",
      "Iteration: 120 of 277\ttrain_loss: 1.9888\n",
      "Iteration: 122 of 277\ttrain_loss: 2.0683\n",
      "Iteration: 124 of 277\ttrain_loss: 2.1423\n",
      "Iteration: 126 of 277\ttrain_loss: 2.1520\n",
      "Iteration: 128 of 277\ttrain_loss: 1.9499\n",
      "Iteration: 130 of 277\ttrain_loss: 1.8988\n",
      "Iteration: 132 of 277\ttrain_loss: 2.0146\n",
      "Iteration: 134 of 277\ttrain_loss: 2.0116\n",
      "Iteration: 136 of 277\ttrain_loss: 1.9723\n",
      "Iteration: 138 of 277\ttrain_loss: 2.1479\n",
      "Iteration: 140 of 277\ttrain_loss: 1.9050\n",
      "Iteration: 142 of 277\ttrain_loss: 1.9324\n",
      "Iteration: 144 of 277\ttrain_loss: 2.0276\n",
      "Iteration: 146 of 277\ttrain_loss: 2.0042\n",
      "Iteration: 148 of 277\ttrain_loss: 1.9986\n",
      "Iteration: 150 of 277\ttrain_loss: 2.0264\n",
      "Iteration: 152 of 277\ttrain_loss: 2.0461\n",
      "Iteration: 154 of 277\ttrain_loss: 1.9777\n",
      "Iteration: 156 of 277\ttrain_loss: 1.9492\n",
      "Iteration: 158 of 277\ttrain_loss: 2.2877\n",
      "Iteration: 160 of 277\ttrain_loss: 2.0601\n",
      "Iteration: 162 of 277\ttrain_loss: 2.0750\n",
      "Iteration: 164 of 277\ttrain_loss: 1.9479\n",
      "Iteration: 166 of 277\ttrain_loss: 2.0227\n",
      "Iteration: 168 of 277\ttrain_loss: 2.1277\n",
      "Iteration: 170 of 277\ttrain_loss: 1.9871\n",
      "Iteration: 172 of 277\ttrain_loss: 2.0058\n",
      "Iteration: 174 of 277\ttrain_loss: 2.0392\n",
      "Iteration: 176 of 277\ttrain_loss: 2.0427\n",
      "Iteration: 178 of 277\ttrain_loss: 1.9709\n",
      "Iteration: 180 of 277\ttrain_loss: 2.1537\n",
      "Iteration: 182 of 277\ttrain_loss: 2.1215\n",
      "Iteration: 184 of 277\ttrain_loss: 2.0890\n",
      "Iteration: 186 of 277\ttrain_loss: 2.0429\n",
      "Iteration: 188 of 277\ttrain_loss: 2.2385\n",
      "Iteration: 190 of 277\ttrain_loss: 2.0380\n",
      "Iteration: 192 of 277\ttrain_loss: 1.9739\n",
      "Iteration: 194 of 277\ttrain_loss: 1.8539\n",
      "Iteration: 196 of 277\ttrain_loss: 2.0379\n",
      "Iteration: 198 of 277\ttrain_loss: 1.9365\n",
      "Iteration: 200 of 277\ttrain_loss: 2.1079\n",
      "Iteration: 202 of 277\ttrain_loss: 2.1123\n",
      "Iteration: 204 of 277\ttrain_loss: 2.0096\n",
      "Iteration: 206 of 277\ttrain_loss: 2.1759\n",
      "Iteration: 208 of 277\ttrain_loss: 1.9902\n",
      "Iteration: 210 of 277\ttrain_loss: 2.0433\n",
      "Iteration: 212 of 277\ttrain_loss: 1.9028\n",
      "Iteration: 214 of 277\ttrain_loss: 2.0496\n",
      "Iteration: 216 of 277\ttrain_loss: 2.0867\n",
      "Iteration: 218 of 277\ttrain_loss: 1.8691\n",
      "Iteration: 220 of 277\ttrain_loss: 2.0945\n",
      "Iteration: 222 of 277\ttrain_loss: 1.9303\n",
      "Iteration: 224 of 277\ttrain_loss: 2.0046\n",
      "Iteration: 226 of 277\ttrain_loss: 2.0104\n",
      "Iteration: 228 of 277\ttrain_loss: 2.0387\n",
      "Iteration: 230 of 277\ttrain_loss: 2.0347\n",
      "Iteration: 232 of 277\ttrain_loss: 1.9321\n",
      "Iteration: 234 of 277\ttrain_loss: 2.0159\n",
      "Iteration: 236 of 277\ttrain_loss: 1.9590\n",
      "Iteration: 238 of 277\ttrain_loss: 2.1729\n",
      "Iteration: 240 of 277\ttrain_loss: 2.1393\n",
      "Iteration: 242 of 277\ttrain_loss: 2.2394\n",
      "Iteration: 244 of 277\ttrain_loss: 2.0741\n",
      "Iteration: 246 of 277\ttrain_loss: 2.1862\n",
      "Iteration: 248 of 277\ttrain_loss: 1.9596\n",
      "Iteration: 250 of 277\ttrain_loss: 2.0911\n",
      "Iteration: 252 of 277\ttrain_loss: 2.3034\n",
      "Iteration: 254 of 277\ttrain_loss: 2.1666\n",
      "Iteration: 256 of 277\ttrain_loss: 1.9778\n",
      "Iteration: 258 of 277\ttrain_loss: 2.3241\n",
      "Iteration: 260 of 277\ttrain_loss: 2.0413\n",
      "Iteration: 262 of 277\ttrain_loss: 1.9583\n",
      "Iteration: 264 of 277\ttrain_loss: 2.0782\n",
      "Iteration: 266 of 277\ttrain_loss: 2.0296\n",
      "Iteration: 268 of 277\ttrain_loss: 1.8571\n",
      "Iteration: 270 of 277\ttrain_loss: 2.1334\n",
      "Iteration: 272 of 277\ttrain_loss: 2.2278\n",
      "Iteration: 274 of 277\ttrain_loss: 2.0312\n",
      "Iteration: 276 of 277\ttrain_loss: 2.1142\n",
      "Iteration: 277 of 277\ttrain_loss: 1.9500\n",
      "Average Score for this Epoch: 2.041496992111206\n",
      "Eval_loss: 2.669313907623291\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 11 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 2.0538\n",
      "Iteration: 2 of 277\ttrain_loss: 1.9307\n",
      "Iteration: 4 of 277\ttrain_loss: 1.9074\n",
      "Iteration: 6 of 277\ttrain_loss: 1.9337\n",
      "Iteration: 8 of 277\ttrain_loss: 2.0431\n",
      "Iteration: 10 of 277\ttrain_loss: 1.8697\n",
      "Iteration: 12 of 277\ttrain_loss: 1.9816\n",
      "Iteration: 14 of 277\ttrain_loss: 1.8737\n",
      "Iteration: 16 of 277\ttrain_loss: 1.9779\n",
      "Iteration: 18 of 277\ttrain_loss: 1.8298\n",
      "Iteration: 20 of 277\ttrain_loss: 2.0261\n",
      "Iteration: 22 of 277\ttrain_loss: 2.0765\n",
      "Iteration: 24 of 277\ttrain_loss: 1.8804\n",
      "Iteration: 26 of 277\ttrain_loss: 1.9846\n",
      "Iteration: 28 of 277\ttrain_loss: 2.1410\n",
      "Iteration: 30 of 277\ttrain_loss: 1.8820\n",
      "Iteration: 32 of 277\ttrain_loss: 1.9738\n",
      "Iteration: 34 of 277\ttrain_loss: 1.9934\n",
      "Iteration: 36 of 277\ttrain_loss: 2.0769\n",
      "Iteration: 38 of 277\ttrain_loss: 2.0128\n",
      "Iteration: 40 of 277\ttrain_loss: 2.0438\n",
      "Iteration: 42 of 277\ttrain_loss: 1.8326\n",
      "Iteration: 44 of 277\ttrain_loss: 2.0469\n",
      "Iteration: 46 of 277\ttrain_loss: 1.9238\n",
      "Iteration: 48 of 277\ttrain_loss: 2.0299\n",
      "Iteration: 50 of 277\ttrain_loss: 1.8960\n",
      "Iteration: 52 of 277\ttrain_loss: 2.1006\n",
      "Iteration: 54 of 277\ttrain_loss: 2.0531\n",
      "Iteration: 56 of 277\ttrain_loss: 1.9616\n",
      "Iteration: 58 of 277\ttrain_loss: 1.8771\n",
      "Iteration: 60 of 277\ttrain_loss: 1.9808\n",
      "Iteration: 62 of 277\ttrain_loss: 1.8628\n",
      "Iteration: 64 of 277\ttrain_loss: 1.9928\n",
      "Iteration: 66 of 277\ttrain_loss: 1.9247\n",
      "Iteration: 68 of 277\ttrain_loss: 2.1893\n",
      "Iteration: 70 of 277\ttrain_loss: 2.0584\n",
      "Iteration: 72 of 277\ttrain_loss: 1.9045\n",
      "Iteration: 74 of 277\ttrain_loss: 1.9254\n",
      "Iteration: 76 of 277\ttrain_loss: 2.1251\n",
      "Iteration: 78 of 277\ttrain_loss: 1.8835\n",
      "Iteration: 80 of 277\ttrain_loss: 1.9951\n",
      "Iteration: 82 of 277\ttrain_loss: 2.0910\n",
      "Iteration: 84 of 277\ttrain_loss: 1.9773\n",
      "Iteration: 86 of 277\ttrain_loss: 2.1131\n",
      "Iteration: 88 of 277\ttrain_loss: 2.0225\n",
      "Iteration: 90 of 277\ttrain_loss: 2.0114\n",
      "Iteration: 92 of 277\ttrain_loss: 2.2022\n",
      "Iteration: 94 of 277\ttrain_loss: 2.0044\n",
      "Iteration: 96 of 277\ttrain_loss: 2.1124\n",
      "Iteration: 98 of 277\ttrain_loss: 2.0806\n",
      "Iteration: 100 of 277\ttrain_loss: 1.9610\n",
      "Iteration: 102 of 277\ttrain_loss: 2.0923\n",
      "Iteration: 104 of 277\ttrain_loss: 2.2689\n",
      "Iteration: 106 of 277\ttrain_loss: 2.0394\n",
      "Iteration: 108 of 277\ttrain_loss: 1.8494\n",
      "Iteration: 110 of 277\ttrain_loss: 2.1028\n",
      "Iteration: 112 of 277\ttrain_loss: 1.9170\n",
      "Iteration: 114 of 277\ttrain_loss: 2.0504\n",
      "Iteration: 116 of 277\ttrain_loss: 2.0669\n",
      "Iteration: 118 of 277\ttrain_loss: 2.0400\n",
      "Iteration: 120 of 277\ttrain_loss: 2.0758\n",
      "Iteration: 122 of 277\ttrain_loss: 2.0105\n",
      "Iteration: 124 of 277\ttrain_loss: 2.1274\n",
      "Iteration: 126 of 277\ttrain_loss: 1.8242\n",
      "Iteration: 128 of 277\ttrain_loss: 2.0240\n",
      "Iteration: 130 of 277\ttrain_loss: 2.0309\n",
      "Iteration: 132 of 277\ttrain_loss: 2.2596\n",
      "Iteration: 134 of 277\ttrain_loss: 1.8941\n",
      "Iteration: 136 of 277\ttrain_loss: 2.1480\n",
      "Iteration: 138 of 277\ttrain_loss: 1.8500\n",
      "Iteration: 140 of 277\ttrain_loss: 2.1205\n",
      "Iteration: 142 of 277\ttrain_loss: 2.0732\n",
      "Iteration: 144 of 277\ttrain_loss: 2.0251\n",
      "Iteration: 146 of 277\ttrain_loss: 2.0064\n",
      "Iteration: 148 of 277\ttrain_loss: 1.8971\n",
      "Iteration: 150 of 277\ttrain_loss: 1.9969\n",
      "Iteration: 152 of 277\ttrain_loss: 2.0524\n",
      "Iteration: 154 of 277\ttrain_loss: 2.5371\n",
      "Iteration: 156 of 277\ttrain_loss: 2.0494\n",
      "Iteration: 158 of 277\ttrain_loss: 2.0343\n",
      "Iteration: 160 of 277\ttrain_loss: 1.9860\n",
      "Iteration: 162 of 277\ttrain_loss: 1.9442\n",
      "Iteration: 164 of 277\ttrain_loss: 2.1576\n",
      "Iteration: 166 of 277\ttrain_loss: 2.1052\n",
      "Iteration: 168 of 277\ttrain_loss: 1.9327\n",
      "Iteration: 170 of 277\ttrain_loss: 2.0814\n",
      "Iteration: 172 of 277\ttrain_loss: 2.0351\n",
      "Iteration: 174 of 277\ttrain_loss: 2.0903\n",
      "Iteration: 176 of 277\ttrain_loss: 2.0594\n",
      "Iteration: 178 of 277\ttrain_loss: 2.1675\n",
      "Iteration: 180 of 277\ttrain_loss: 2.1694\n",
      "Iteration: 182 of 277\ttrain_loss: 1.8480\n",
      "Iteration: 184 of 277\ttrain_loss: 1.9529\n",
      "Iteration: 186 of 277\ttrain_loss: 2.0973\n",
      "Iteration: 188 of 277\ttrain_loss: 2.0241\n",
      "Iteration: 190 of 277\ttrain_loss: 2.1429\n",
      "Iteration: 192 of 277\ttrain_loss: 2.0663\n",
      "Iteration: 194 of 277\ttrain_loss: 2.1028\n",
      "Iteration: 196 of 277\ttrain_loss: 2.1145\n",
      "Iteration: 198 of 277\ttrain_loss: 2.1587\n",
      "Iteration: 200 of 277\ttrain_loss: 2.1251\n",
      "Iteration: 202 of 277\ttrain_loss: 2.1310\n",
      "Iteration: 204 of 277\ttrain_loss: 2.1987\n",
      "Iteration: 206 of 277\ttrain_loss: 1.8671\n",
      "Iteration: 208 of 277\ttrain_loss: 2.1234\n",
      "Iteration: 210 of 277\ttrain_loss: 2.1512\n",
      "Iteration: 212 of 277\ttrain_loss: 1.9984\n",
      "Iteration: 214 of 277\ttrain_loss: 1.9582\n",
      "Iteration: 216 of 277\ttrain_loss: 2.1084\n",
      "Iteration: 218 of 277\ttrain_loss: 2.1746\n",
      "Iteration: 220 of 277\ttrain_loss: 2.0849\n",
      "Iteration: 222 of 277\ttrain_loss: 2.0134\n",
      "Iteration: 224 of 277\ttrain_loss: 2.0756\n",
      "Iteration: 226 of 277\ttrain_loss: 2.0915\n",
      "Iteration: 228 of 277\ttrain_loss: 1.9279\n",
      "Iteration: 230 of 277\ttrain_loss: 2.1110\n",
      "Iteration: 232 of 277\ttrain_loss: 1.9403\n",
      "Iteration: 234 of 277\ttrain_loss: 1.9866\n",
      "Iteration: 236 of 277\ttrain_loss: 2.1850\n",
      "Iteration: 238 of 277\ttrain_loss: 1.9949\n",
      "Iteration: 240 of 277\ttrain_loss: 2.0395\n",
      "Iteration: 242 of 277\ttrain_loss: 2.1307\n",
      "Iteration: 244 of 277\ttrain_loss: 2.1162\n",
      "Iteration: 246 of 277\ttrain_loss: 1.8942\n",
      "Iteration: 248 of 277\ttrain_loss: 2.2454\n",
      "Iteration: 250 of 277\ttrain_loss: 1.8045\n",
      "Iteration: 252 of 277\ttrain_loss: 2.0959\n",
      "Iteration: 254 of 277\ttrain_loss: 2.0933\n",
      "Iteration: 256 of 277\ttrain_loss: 2.0810\n",
      "Iteration: 258 of 277\ttrain_loss: 2.1041\n",
      "Iteration: 260 of 277\ttrain_loss: 2.0970\n",
      "Iteration: 262 of 277\ttrain_loss: 2.2244\n",
      "Iteration: 264 of 277\ttrain_loss: 2.0905\n",
      "Iteration: 266 of 277\ttrain_loss: 2.0821\n",
      "Iteration: 268 of 277\ttrain_loss: 2.1095\n",
      "Iteration: 270 of 277\ttrain_loss: 2.1800\n",
      "Iteration: 272 of 277\ttrain_loss: 2.1625\n",
      "Iteration: 274 of 277\ttrain_loss: 2.1196\n",
      "Iteration: 276 of 277\ttrain_loss: 2.0348\n",
      "Iteration: 277 of 277\ttrain_loss: 1.9687\n",
      "Average Score for this Epoch: 2.0401227474212646\n",
      "Eval_loss: 2.660457134246826\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 12 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.8995\n",
      "Iteration: 2 of 277\ttrain_loss: 1.8444\n",
      "Iteration: 4 of 277\ttrain_loss: 1.9767\n",
      "Iteration: 6 of 277\ttrain_loss: 1.9843\n",
      "Iteration: 8 of 277\ttrain_loss: 2.0662\n",
      "Iteration: 10 of 277\ttrain_loss: 1.8892\n",
      "Iteration: 12 of 277\ttrain_loss: 1.9492\n",
      "Iteration: 14 of 277\ttrain_loss: 2.0045\n",
      "Iteration: 16 of 277\ttrain_loss: 2.0894\n",
      "Iteration: 18 of 277\ttrain_loss: 1.9388\n",
      "Iteration: 20 of 277\ttrain_loss: 1.8852\n",
      "Iteration: 22 of 277\ttrain_loss: 1.7915\n",
      "Iteration: 24 of 277\ttrain_loss: 1.8894\n",
      "Iteration: 26 of 277\ttrain_loss: 1.9822\n",
      "Iteration: 28 of 277\ttrain_loss: 1.9333\n",
      "Iteration: 30 of 277\ttrain_loss: 1.8640\n",
      "Iteration: 32 of 277\ttrain_loss: 1.9316\n",
      "Iteration: 34 of 277\ttrain_loss: 1.9268\n",
      "Iteration: 36 of 277\ttrain_loss: 1.9412\n",
      "Iteration: 38 of 277\ttrain_loss: 1.8039\n",
      "Iteration: 40 of 277\ttrain_loss: 2.0652\n",
      "Iteration: 42 of 277\ttrain_loss: 1.8067\n",
      "Iteration: 44 of 277\ttrain_loss: 1.9123\n",
      "Iteration: 46 of 277\ttrain_loss: 1.9309\n",
      "Iteration: 48 of 277\ttrain_loss: 2.0701\n",
      "Iteration: 50 of 277\ttrain_loss: 2.1186\n",
      "Iteration: 52 of 277\ttrain_loss: 2.0769\n",
      "Iteration: 54 of 277\ttrain_loss: 1.9668\n",
      "Iteration: 56 of 277\ttrain_loss: 1.9518\n",
      "Iteration: 58 of 277\ttrain_loss: 1.9519\n",
      "Iteration: 60 of 277\ttrain_loss: 1.9336\n",
      "Iteration: 62 of 277\ttrain_loss: 2.0398\n",
      "Iteration: 64 of 277\ttrain_loss: 1.9377\n",
      "Iteration: 66 of 277\ttrain_loss: 2.0336\n",
      "Iteration: 68 of 277\ttrain_loss: 1.9319\n",
      "Iteration: 70 of 277\ttrain_loss: 1.8785\n",
      "Iteration: 72 of 277\ttrain_loss: 1.9896\n",
      "Iteration: 74 of 277\ttrain_loss: 1.9243\n",
      "Iteration: 76 of 277\ttrain_loss: 2.1070\n",
      "Iteration: 78 of 277\ttrain_loss: 1.9572\n",
      "Iteration: 80 of 277\ttrain_loss: 2.0757\n",
      "Iteration: 82 of 277\ttrain_loss: 2.0836\n",
      "Iteration: 84 of 277\ttrain_loss: 1.9153\n",
      "Iteration: 86 of 277\ttrain_loss: 1.9085\n",
      "Iteration: 88 of 277\ttrain_loss: 1.9813\n",
      "Iteration: 90 of 277\ttrain_loss: 2.0857\n",
      "Iteration: 92 of 277\ttrain_loss: 1.7580\n",
      "Iteration: 94 of 277\ttrain_loss: 2.0999\n",
      "Iteration: 96 of 277\ttrain_loss: 1.6126\n",
      "Iteration: 98 of 277\ttrain_loss: 2.1053\n",
      "Iteration: 100 of 277\ttrain_loss: 2.2016\n",
      "Iteration: 102 of 277\ttrain_loss: 2.2015\n",
      "Iteration: 104 of 277\ttrain_loss: 1.9886\n",
      "Iteration: 106 of 277\ttrain_loss: 1.9723\n",
      "Iteration: 108 of 277\ttrain_loss: 2.0176\n",
      "Iteration: 110 of 277\ttrain_loss: 2.0989\n",
      "Iteration: 112 of 277\ttrain_loss: 2.2048\n",
      "Iteration: 114 of 277\ttrain_loss: 1.9947\n",
      "Iteration: 116 of 277\ttrain_loss: 2.0800\n",
      "Iteration: 118 of 277\ttrain_loss: 1.9929\n",
      "Iteration: 120 of 277\ttrain_loss: 2.0890\n",
      "Iteration: 122 of 277\ttrain_loss: 1.9463\n",
      "Iteration: 124 of 277\ttrain_loss: 1.9463\n",
      "Iteration: 126 of 277\ttrain_loss: 2.0895\n",
      "Iteration: 128 of 277\ttrain_loss: 2.1558\n",
      "Iteration: 130 of 277\ttrain_loss: 2.3446\n",
      "Iteration: 132 of 277\ttrain_loss: 2.0823\n",
      "Iteration: 134 of 277\ttrain_loss: 1.9137\n",
      "Iteration: 136 of 277\ttrain_loss: 1.9774\n",
      "Iteration: 138 of 277\ttrain_loss: 1.9536\n",
      "Iteration: 140 of 277\ttrain_loss: 1.8227\n",
      "Iteration: 142 of 277\ttrain_loss: 1.9550\n",
      "Iteration: 144 of 277\ttrain_loss: 1.9369\n",
      "Iteration: 146 of 277\ttrain_loss: 2.2933\n",
      "Iteration: 148 of 277\ttrain_loss: 1.9549\n",
      "Iteration: 150 of 277\ttrain_loss: 2.0823\n",
      "Iteration: 152 of 277\ttrain_loss: 2.0192\n",
      "Iteration: 154 of 277\ttrain_loss: 1.9889\n",
      "Iteration: 156 of 277\ttrain_loss: 1.9581\n",
      "Iteration: 158 of 277\ttrain_loss: 2.0130\n",
      "Iteration: 160 of 277\ttrain_loss: 1.9459\n",
      "Iteration: 162 of 277\ttrain_loss: 2.7740\n",
      "Iteration: 164 of 277\ttrain_loss: 2.1267\n",
      "Iteration: 166 of 277\ttrain_loss: 1.9654\n",
      "Iteration: 168 of 277\ttrain_loss: 1.9713\n",
      "Iteration: 170 of 277\ttrain_loss: 2.0435\n",
      "Iteration: 172 of 277\ttrain_loss: 2.1349\n",
      "Iteration: 174 of 277\ttrain_loss: 2.0416\n",
      "Iteration: 176 of 277\ttrain_loss: 2.0593\n",
      "Iteration: 178 of 277\ttrain_loss: 2.1649\n",
      "Iteration: 180 of 277\ttrain_loss: 2.1532\n",
      "Iteration: 182 of 277\ttrain_loss: 2.0632\n",
      "Iteration: 184 of 277\ttrain_loss: 1.8984\n",
      "Iteration: 186 of 277\ttrain_loss: 2.0823\n",
      "Iteration: 188 of 277\ttrain_loss: 2.1811\n",
      "Iteration: 190 of 277\ttrain_loss: 2.0014\n",
      "Iteration: 192 of 277\ttrain_loss: 2.0369\n",
      "Iteration: 194 of 277\ttrain_loss: 2.1198\n",
      "Iteration: 196 of 277\ttrain_loss: 2.1872\n",
      "Iteration: 198 of 277\ttrain_loss: 2.1495\n",
      "Iteration: 200 of 277\ttrain_loss: 2.0907\n",
      "Iteration: 202 of 277\ttrain_loss: 1.8615\n",
      "Iteration: 204 of 277\ttrain_loss: 2.0113\n",
      "Iteration: 206 of 277\ttrain_loss: 2.2007\n",
      "Iteration: 208 of 277\ttrain_loss: 2.1424\n",
      "Iteration: 210 of 277\ttrain_loss: 2.1330\n",
      "Iteration: 212 of 277\ttrain_loss: 2.1580\n",
      "Iteration: 214 of 277\ttrain_loss: 2.0666\n",
      "Iteration: 216 of 277\ttrain_loss: 1.9781\n",
      "Iteration: 218 of 277\ttrain_loss: 2.3019\n",
      "Iteration: 220 of 277\ttrain_loss: 2.0377\n",
      "Iteration: 222 of 277\ttrain_loss: 1.8875\n",
      "Iteration: 224 of 277\ttrain_loss: 2.0644\n",
      "Iteration: 226 of 277\ttrain_loss: 2.1301\n",
      "Iteration: 228 of 277\ttrain_loss: 1.8693\n",
      "Iteration: 230 of 277\ttrain_loss: 2.1022\n",
      "Iteration: 232 of 277\ttrain_loss: 2.0867\n",
      "Iteration: 234 of 277\ttrain_loss: 2.1097\n",
      "Iteration: 236 of 277\ttrain_loss: 1.9632\n",
      "Iteration: 238 of 277\ttrain_loss: 2.2010\n",
      "Iteration: 240 of 277\ttrain_loss: 2.1510\n",
      "Iteration: 242 of 277\ttrain_loss: 1.9354\n",
      "Iteration: 244 of 277\ttrain_loss: 2.0079\n",
      "Iteration: 246 of 277\ttrain_loss: 1.9271\n",
      "Iteration: 248 of 277\ttrain_loss: 2.1412\n",
      "Iteration: 250 of 277\ttrain_loss: 2.1817\n",
      "Iteration: 252 of 277\ttrain_loss: 1.9024\n",
      "Iteration: 254 of 277\ttrain_loss: 2.0749\n",
      "Iteration: 256 of 277\ttrain_loss: 2.1712\n",
      "Iteration: 258 of 277\ttrain_loss: 1.9937\n",
      "Iteration: 260 of 277\ttrain_loss: 1.9771\n",
      "Iteration: 262 of 277\ttrain_loss: 1.8856\n",
      "Iteration: 264 of 277\ttrain_loss: 1.8660\n",
      "Iteration: 266 of 277\ttrain_loss: 1.9475\n",
      "Iteration: 268 of 277\ttrain_loss: 2.3048\n",
      "Iteration: 270 of 277\ttrain_loss: 2.1767\n",
      "Iteration: 272 of 277\ttrain_loss: 2.2879\n",
      "Iteration: 274 of 277\ttrain_loss: 2.0206\n",
      "Iteration: 276 of 277\ttrain_loss: 2.0393\n",
      "Iteration: 277 of 277\ttrain_loss: 1.9400\n",
      "Average Score for this Epoch: 2.018951177597046\n",
      "Eval_loss: 2.6379024982452393\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 13 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.8956\n",
      "Iteration: 2 of 277\ttrain_loss: 1.8889\n",
      "Iteration: 4 of 277\ttrain_loss: 1.9216\n",
      "Iteration: 6 of 277\ttrain_loss: 1.7232\n",
      "Iteration: 8 of 277\ttrain_loss: 1.7368\n",
      "Iteration: 10 of 277\ttrain_loss: 1.9741\n",
      "Iteration: 12 of 277\ttrain_loss: 1.8998\n",
      "Iteration: 14 of 277\ttrain_loss: 1.8376\n",
      "Iteration: 16 of 277\ttrain_loss: 1.9123\n",
      "Iteration: 18 of 277\ttrain_loss: 1.9470\n",
      "Iteration: 20 of 277\ttrain_loss: 1.9043\n",
      "Iteration: 22 of 277\ttrain_loss: 1.9857\n",
      "Iteration: 24 of 277\ttrain_loss: 1.9157\n",
      "Iteration: 26 of 277\ttrain_loss: 1.6382\n",
      "Iteration: 28 of 277\ttrain_loss: 1.7622\n",
      "Iteration: 30 of 277\ttrain_loss: 1.7995\n",
      "Iteration: 32 of 277\ttrain_loss: 1.7422\n",
      "Iteration: 34 of 277\ttrain_loss: 1.8084\n",
      "Iteration: 36 of 277\ttrain_loss: 1.7561\n",
      "Iteration: 38 of 277\ttrain_loss: 1.8256\n",
      "Iteration: 40 of 277\ttrain_loss: 1.7695\n",
      "Iteration: 42 of 277\ttrain_loss: 1.9343\n",
      "Iteration: 44 of 277\ttrain_loss: 2.2127\n",
      "Iteration: 46 of 277\ttrain_loss: 1.8695\n",
      "Iteration: 48 of 277\ttrain_loss: 1.8243\n",
      "Iteration: 50 of 277\ttrain_loss: 1.7840\n",
      "Iteration: 52 of 277\ttrain_loss: 1.7255\n",
      "Iteration: 54 of 277\ttrain_loss: 1.8753\n",
      "Iteration: 56 of 277\ttrain_loss: 1.8214\n",
      "Iteration: 58 of 277\ttrain_loss: 2.0446\n",
      "Iteration: 60 of 277\ttrain_loss: 1.7616\n",
      "Iteration: 62 of 277\ttrain_loss: 1.6880\n",
      "Iteration: 64 of 277\ttrain_loss: 1.7292\n",
      "Iteration: 66 of 277\ttrain_loss: 1.8226\n",
      "Iteration: 68 of 277\ttrain_loss: 1.8354\n",
      "Iteration: 70 of 277\ttrain_loss: 1.8827\n",
      "Iteration: 72 of 277\ttrain_loss: 1.9589\n",
      "Iteration: 74 of 277\ttrain_loss: 1.7806\n",
      "Iteration: 76 of 277\ttrain_loss: 2.1323\n",
      "Iteration: 78 of 277\ttrain_loss: 1.8587\n",
      "Iteration: 80 of 277\ttrain_loss: 1.7784\n",
      "Iteration: 82 of 277\ttrain_loss: 1.9184\n",
      "Iteration: 84 of 277\ttrain_loss: 1.8387\n",
      "Iteration: 86 of 277\ttrain_loss: 1.8437\n",
      "Iteration: 88 of 277\ttrain_loss: 1.8886\n",
      "Iteration: 90 of 277\ttrain_loss: 1.8231\n",
      "Iteration: 92 of 277\ttrain_loss: 1.7243\n",
      "Iteration: 94 of 277\ttrain_loss: 1.8441\n",
      "Iteration: 96 of 277\ttrain_loss: 1.8738\n",
      "Iteration: 98 of 277\ttrain_loss: 1.7139\n",
      "Iteration: 100 of 277\ttrain_loss: 1.7751\n",
      "Iteration: 102 of 277\ttrain_loss: 2.0058\n",
      "Iteration: 104 of 277\ttrain_loss: 1.9249\n",
      "Iteration: 106 of 277\ttrain_loss: 1.8411\n",
      "Iteration: 108 of 277\ttrain_loss: 1.8978\n",
      "Iteration: 110 of 277\ttrain_loss: 1.8233\n",
      "Iteration: 112 of 277\ttrain_loss: 2.0042\n",
      "Iteration: 114 of 277\ttrain_loss: 1.8593\n",
      "Iteration: 116 of 277\ttrain_loss: 1.8456\n",
      "Iteration: 118 of 277\ttrain_loss: 1.8424\n",
      "Iteration: 120 of 277\ttrain_loss: 1.7063\n",
      "Iteration: 122 of 277\ttrain_loss: 1.8661\n",
      "Iteration: 124 of 277\ttrain_loss: 1.7771\n",
      "Iteration: 126 of 277\ttrain_loss: 1.8436\n",
      "Iteration: 128 of 277\ttrain_loss: 1.8072\n",
      "Iteration: 130 of 277\ttrain_loss: 1.8290\n",
      "Iteration: 132 of 277\ttrain_loss: 1.9629\n",
      "Iteration: 134 of 277\ttrain_loss: 1.8796\n",
      "Iteration: 136 of 277\ttrain_loss: 1.7283\n",
      "Iteration: 138 of 277\ttrain_loss: 1.7439\n",
      "Iteration: 140 of 277\ttrain_loss: 1.9151\n",
      "Iteration: 142 of 277\ttrain_loss: 1.9209\n",
      "Iteration: 144 of 277\ttrain_loss: 1.9026\n",
      "Iteration: 146 of 277\ttrain_loss: 1.7978\n",
      "Iteration: 148 of 277\ttrain_loss: 1.8954\n",
      "Iteration: 150 of 277\ttrain_loss: 1.8392\n",
      "Iteration: 152 of 277\ttrain_loss: 1.7307\n",
      "Iteration: 154 of 277\ttrain_loss: 1.8224\n",
      "Iteration: 156 of 277\ttrain_loss: 1.9221\n",
      "Iteration: 158 of 277\ttrain_loss: 1.8570\n",
      "Iteration: 160 of 277\ttrain_loss: 1.7581\n",
      "Iteration: 162 of 277\ttrain_loss: 1.8850\n",
      "Iteration: 164 of 277\ttrain_loss: 1.7923\n",
      "Iteration: 166 of 277\ttrain_loss: 1.9614\n",
      "Iteration: 168 of 277\ttrain_loss: 1.8864\n",
      "Iteration: 170 of 277\ttrain_loss: 1.8122\n",
      "Iteration: 172 of 277\ttrain_loss: 1.8420\n",
      "Iteration: 174 of 277\ttrain_loss: 1.7121\n",
      "Iteration: 176 of 277\ttrain_loss: 1.6810\n",
      "Iteration: 178 of 277\ttrain_loss: 1.8223\n",
      "Iteration: 180 of 277\ttrain_loss: 1.6320\n",
      "Iteration: 182 of 277\ttrain_loss: 1.8642\n",
      "Iteration: 184 of 277\ttrain_loss: 1.8965\n",
      "Iteration: 186 of 277\ttrain_loss: 1.8778\n",
      "Iteration: 188 of 277\ttrain_loss: 1.8302\n",
      "Iteration: 190 of 277\ttrain_loss: 1.8141\n",
      "Iteration: 192 of 277\ttrain_loss: 1.7816\n",
      "Iteration: 194 of 277\ttrain_loss: 1.9206\n",
      "Iteration: 196 of 277\ttrain_loss: 2.0051\n",
      "Iteration: 198 of 277\ttrain_loss: 1.9742\n",
      "Iteration: 200 of 277\ttrain_loss: 1.7820\n",
      "Iteration: 202 of 277\ttrain_loss: 1.9052\n",
      "Iteration: 204 of 277\ttrain_loss: 1.7977\n",
      "Iteration: 206 of 277\ttrain_loss: 1.6815\n",
      "Iteration: 208 of 277\ttrain_loss: 1.8866\n",
      "Iteration: 210 of 277\ttrain_loss: 1.9295\n",
      "Iteration: 212 of 277\ttrain_loss: 1.7631\n",
      "Iteration: 214 of 277\ttrain_loss: 1.6410\n",
      "Iteration: 216 of 277\ttrain_loss: 1.9478\n",
      "Iteration: 218 of 277\ttrain_loss: 1.8344\n",
      "Iteration: 220 of 277\ttrain_loss: 1.9317\n",
      "Iteration: 222 of 277\ttrain_loss: 1.7161\n",
      "Iteration: 224 of 277\ttrain_loss: 1.8446\n",
      "Iteration: 226 of 277\ttrain_loss: 2.0459\n",
      "Iteration: 228 of 277\ttrain_loss: 1.7532\n",
      "Iteration: 230 of 277\ttrain_loss: 1.8988\n",
      "Iteration: 232 of 277\ttrain_loss: 1.7380\n",
      "Iteration: 234 of 277\ttrain_loss: 1.9326\n",
      "Iteration: 236 of 277\ttrain_loss: 1.8917\n",
      "Iteration: 238 of 277\ttrain_loss: 2.0217\n",
      "Iteration: 240 of 277\ttrain_loss: 1.7384\n",
      "Iteration: 242 of 277\ttrain_loss: 1.9822\n",
      "Iteration: 244 of 277\ttrain_loss: 1.8776\n",
      "Iteration: 246 of 277\ttrain_loss: 1.8709\n",
      "Iteration: 248 of 277\ttrain_loss: 1.9795\n",
      "Iteration: 250 of 277\ttrain_loss: 1.8794\n",
      "Iteration: 252 of 277\ttrain_loss: 1.8298\n",
      "Iteration: 254 of 277\ttrain_loss: 1.9213\n",
      "Iteration: 256 of 277\ttrain_loss: 1.7953\n",
      "Iteration: 258 of 277\ttrain_loss: 1.9658\n",
      "Iteration: 260 of 277\ttrain_loss: 1.9522\n",
      "Iteration: 262 of 277\ttrain_loss: 1.6530\n",
      "Iteration: 264 of 277\ttrain_loss: 1.9322\n",
      "Iteration: 266 of 277\ttrain_loss: 1.7035\n",
      "Iteration: 268 of 277\ttrain_loss: 1.9375\n",
      "Iteration: 270 of 277\ttrain_loss: 1.8695\n",
      "Iteration: 272 of 277\ttrain_loss: 1.8751\n",
      "Iteration: 274 of 277\ttrain_loss: 1.8297\n",
      "Iteration: 276 of 277\ttrain_loss: 2.5434\n",
      "Iteration: 277 of 277\ttrain_loss: 1.7868\n",
      "Average Score for this Epoch: 1.8613066673278809\n",
      "Eval_loss: 2.560102939605713\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 14 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.7158\n",
      "Iteration: 2 of 277\ttrain_loss: 1.7156\n",
      "Iteration: 4 of 277\ttrain_loss: 1.5410\n",
      "Iteration: 6 of 277\ttrain_loss: 1.6949\n",
      "Iteration: 8 of 277\ttrain_loss: 1.6391\n",
      "Iteration: 10 of 277\ttrain_loss: 1.8507\n",
      "Iteration: 12 of 277\ttrain_loss: 1.6890\n",
      "Iteration: 14 of 277\ttrain_loss: 1.7577\n",
      "Iteration: 16 of 277\ttrain_loss: 1.7018\n",
      "Iteration: 18 of 277\ttrain_loss: 1.7830\n",
      "Iteration: 20 of 277\ttrain_loss: 1.6641\n",
      "Iteration: 22 of 277\ttrain_loss: 1.6541\n",
      "Iteration: 24 of 277\ttrain_loss: 1.7008\n",
      "Iteration: 26 of 277\ttrain_loss: 1.6260\n",
      "Iteration: 28 of 277\ttrain_loss: 1.5526\n",
      "Iteration: 30 of 277\ttrain_loss: 1.6451\n",
      "Iteration: 32 of 277\ttrain_loss: 1.6125\n",
      "Iteration: 34 of 277\ttrain_loss: 1.7362\n",
      "Iteration: 36 of 277\ttrain_loss: 1.6023\n",
      "Iteration: 38 of 277\ttrain_loss: 1.6187\n",
      "Iteration: 40 of 277\ttrain_loss: 1.6726\n",
      "Iteration: 42 of 277\ttrain_loss: 1.5356\n",
      "Iteration: 44 of 277\ttrain_loss: 1.6579\n",
      "Iteration: 46 of 277\ttrain_loss: 1.6976\n",
      "Iteration: 48 of 277\ttrain_loss: 1.6966\n",
      "Iteration: 50 of 277\ttrain_loss: 1.6386\n",
      "Iteration: 52 of 277\ttrain_loss: 1.7106\n",
      "Iteration: 54 of 277\ttrain_loss: 1.6941\n",
      "Iteration: 56 of 277\ttrain_loss: 1.7314\n",
      "Iteration: 58 of 277\ttrain_loss: 1.6423\n",
      "Iteration: 60 of 277\ttrain_loss: 1.4559\n",
      "Iteration: 62 of 277\ttrain_loss: 1.6297\n",
      "Iteration: 64 of 277\ttrain_loss: 1.7312\n",
      "Iteration: 66 of 277\ttrain_loss: 1.6635\n",
      "Iteration: 68 of 277\ttrain_loss: 1.7866\n",
      "Iteration: 70 of 277\ttrain_loss: 1.6937\n",
      "Iteration: 72 of 277\ttrain_loss: 1.6745\n",
      "Iteration: 74 of 277\ttrain_loss: 1.7103\n",
      "Iteration: 76 of 277\ttrain_loss: 1.6979\n",
      "Iteration: 78 of 277\ttrain_loss: 1.7153\n",
      "Iteration: 80 of 277\ttrain_loss: 1.5507\n",
      "Iteration: 82 of 277\ttrain_loss: 1.6361\n",
      "Iteration: 84 of 277\ttrain_loss: 1.7912\n",
      "Iteration: 86 of 277\ttrain_loss: 1.4789\n",
      "Iteration: 88 of 277\ttrain_loss: 1.5145\n",
      "Iteration: 90 of 277\ttrain_loss: 1.5678\n",
      "Iteration: 92 of 277\ttrain_loss: 1.6419\n",
      "Iteration: 94 of 277\ttrain_loss: 1.6966\n",
      "Iteration: 96 of 277\ttrain_loss: 1.6050\n",
      "Iteration: 98 of 277\ttrain_loss: 1.6176\n",
      "Iteration: 100 of 277\ttrain_loss: 1.8159\n",
      "Iteration: 102 of 277\ttrain_loss: 1.7277\n",
      "Iteration: 104 of 277\ttrain_loss: 1.6941\n",
      "Iteration: 106 of 277\ttrain_loss: 1.7726\n",
      "Iteration: 108 of 277\ttrain_loss: 1.6248\n",
      "Iteration: 110 of 277\ttrain_loss: 1.7164\n",
      "Iteration: 112 of 277\ttrain_loss: 1.5623\n",
      "Iteration: 114 of 277\ttrain_loss: 1.7676\n",
      "Iteration: 116 of 277\ttrain_loss: 1.6310\n",
      "Iteration: 118 of 277\ttrain_loss: 1.6577\n",
      "Iteration: 120 of 277\ttrain_loss: 1.4964\n",
      "Iteration: 122 of 277\ttrain_loss: 1.6679\n",
      "Iteration: 124 of 277\ttrain_loss: 1.6465\n",
      "Iteration: 126 of 277\ttrain_loss: 1.7585\n",
      "Iteration: 128 of 277\ttrain_loss: 1.7049\n",
      "Iteration: 130 of 277\ttrain_loss: 1.6428\n",
      "Iteration: 132 of 277\ttrain_loss: 1.8768\n",
      "Iteration: 134 of 277\ttrain_loss: 1.6116\n",
      "Iteration: 136 of 277\ttrain_loss: 1.6302\n",
      "Iteration: 138 of 277\ttrain_loss: 1.6739\n",
      "Iteration: 140 of 277\ttrain_loss: 1.7477\n",
      "Iteration: 142 of 277\ttrain_loss: 1.6560\n",
      "Iteration: 144 of 277\ttrain_loss: 1.7864\n",
      "Iteration: 146 of 277\ttrain_loss: 1.6909\n",
      "Iteration: 148 of 277\ttrain_loss: 1.7449\n",
      "Iteration: 150 of 277\ttrain_loss: 1.7109\n",
      "Iteration: 152 of 277\ttrain_loss: 1.5236\n",
      "Iteration: 154 of 277\ttrain_loss: 1.8605\n",
      "Iteration: 156 of 277\ttrain_loss: 1.6635\n",
      "Iteration: 158 of 277\ttrain_loss: 1.6721\n",
      "Iteration: 160 of 277\ttrain_loss: 1.7769\n",
      "Iteration: 162 of 277\ttrain_loss: 1.6985\n",
      "Iteration: 164 of 277\ttrain_loss: 1.5746\n",
      "Iteration: 166 of 277\ttrain_loss: 1.9465\n",
      "Iteration: 168 of 277\ttrain_loss: 1.6139\n",
      "Iteration: 170 of 277\ttrain_loss: 1.8369\n",
      "Iteration: 172 of 277\ttrain_loss: 1.8267\n",
      "Iteration: 174 of 277\ttrain_loss: 1.6774\n",
      "Iteration: 176 of 277\ttrain_loss: 2.2138\n",
      "Iteration: 178 of 277\ttrain_loss: 1.6602\n",
      "Iteration: 180 of 277\ttrain_loss: 1.5779\n",
      "Iteration: 182 of 277\ttrain_loss: 1.7009\n",
      "Iteration: 184 of 277\ttrain_loss: 1.6882\n",
      "Iteration: 186 of 277\ttrain_loss: 1.5732\n",
      "Iteration: 188 of 277\ttrain_loss: 1.6929\n",
      "Iteration: 190 of 277\ttrain_loss: 1.6794\n",
      "Iteration: 192 of 277\ttrain_loss: 1.5880\n",
      "Iteration: 194 of 277\ttrain_loss: 1.8925\n",
      "Iteration: 196 of 277\ttrain_loss: 1.6618\n",
      "Iteration: 198 of 277\ttrain_loss: 1.7621\n",
      "Iteration: 200 of 277\ttrain_loss: 1.8422\n",
      "Iteration: 202 of 277\ttrain_loss: 1.5397\n",
      "Iteration: 204 of 277\ttrain_loss: 1.6789\n",
      "Iteration: 206 of 277\ttrain_loss: 1.9433\n",
      "Iteration: 208 of 277\ttrain_loss: 1.6664\n",
      "Iteration: 210 of 277\ttrain_loss: 1.5960\n",
      "Iteration: 212 of 277\ttrain_loss: 1.5799\n",
      "Iteration: 214 of 277\ttrain_loss: 1.6661\n",
      "Iteration: 216 of 277\ttrain_loss: 1.6213\n",
      "Iteration: 218 of 277\ttrain_loss: 1.6261\n",
      "Iteration: 220 of 277\ttrain_loss: 1.6776\n",
      "Iteration: 222 of 277\ttrain_loss: 1.6762\n",
      "Iteration: 224 of 277\ttrain_loss: 1.5984\n",
      "Iteration: 226 of 277\ttrain_loss: 1.4420\n",
      "Iteration: 228 of 277\ttrain_loss: 1.6839\n",
      "Iteration: 230 of 277\ttrain_loss: 1.7450\n",
      "Iteration: 232 of 277\ttrain_loss: 1.5823\n",
      "Iteration: 234 of 277\ttrain_loss: 1.8736\n",
      "Iteration: 236 of 277\ttrain_loss: 1.6348\n",
      "Iteration: 238 of 277\ttrain_loss: 1.8011\n",
      "Iteration: 240 of 277\ttrain_loss: 1.6258\n",
      "Iteration: 242 of 277\ttrain_loss: 1.6878\n",
      "Iteration: 244 of 277\ttrain_loss: 1.6749\n",
      "Iteration: 246 of 277\ttrain_loss: 1.5741\n",
      "Iteration: 248 of 277\ttrain_loss: 1.5730\n",
      "Iteration: 250 of 277\ttrain_loss: 1.6344\n",
      "Iteration: 252 of 277\ttrain_loss: 1.7211\n",
      "Iteration: 254 of 277\ttrain_loss: 1.6332\n",
      "Iteration: 256 of 277\ttrain_loss: 1.7464\n",
      "Iteration: 258 of 277\ttrain_loss: 1.6836\n",
      "Iteration: 260 of 277\ttrain_loss: 1.7859\n",
      "Iteration: 262 of 277\ttrain_loss: 1.6484\n",
      "Iteration: 264 of 277\ttrain_loss: 1.7132\n",
      "Iteration: 266 of 277\ttrain_loss: 1.6507\n",
      "Iteration: 268 of 277\ttrain_loss: 1.6259\n",
      "Iteration: 270 of 277\ttrain_loss: 1.6363\n",
      "Iteration: 272 of 277\ttrain_loss: 1.7342\n",
      "Iteration: 274 of 277\ttrain_loss: 1.5242\n",
      "Iteration: 276 of 277\ttrain_loss: 1.7573\n",
      "Iteration: 277 of 277\ttrain_loss: 1.6494\n",
      "Average Score for this Epoch: 1.6882514953613281\n",
      "Eval_loss: 2.540705919265747\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 15 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.4847\n",
      "Iteration: 2 of 277\ttrain_loss: 1.5335\n",
      "Iteration: 4 of 277\ttrain_loss: 1.5510\n",
      "Iteration: 6 of 277\ttrain_loss: 1.7236\n",
      "Iteration: 8 of 277\ttrain_loss: 1.5964\n",
      "Iteration: 10 of 277\ttrain_loss: 2.0867\n",
      "Iteration: 12 of 277\ttrain_loss: 1.5422\n",
      "Iteration: 14 of 277\ttrain_loss: 1.5441\n",
      "Iteration: 16 of 277\ttrain_loss: 1.6550\n",
      "Iteration: 18 of 277\ttrain_loss: 1.5372\n",
      "Iteration: 20 of 277\ttrain_loss: 1.6119\n",
      "Iteration: 22 of 277\ttrain_loss: 1.6103\n",
      "Iteration: 24 of 277\ttrain_loss: 1.5560\n",
      "Iteration: 26 of 277\ttrain_loss: 1.4614\n",
      "Iteration: 28 of 277\ttrain_loss: 1.5593\n",
      "Iteration: 30 of 277\ttrain_loss: 1.3499\n",
      "Iteration: 32 of 277\ttrain_loss: 1.6847\n",
      "Iteration: 34 of 277\ttrain_loss: 1.6685\n",
      "Iteration: 36 of 277\ttrain_loss: 1.5457\n",
      "Iteration: 38 of 277\ttrain_loss: 1.6001\n",
      "Iteration: 40 of 277\ttrain_loss: 1.6310\n",
      "Iteration: 42 of 277\ttrain_loss: 1.4991\n",
      "Iteration: 44 of 277\ttrain_loss: 1.5271\n",
      "Iteration: 46 of 277\ttrain_loss: 1.7535\n",
      "Iteration: 48 of 277\ttrain_loss: 1.5360\n",
      "Iteration: 50 of 277\ttrain_loss: 1.6382\n",
      "Iteration: 52 of 277\ttrain_loss: 1.5623\n",
      "Iteration: 54 of 277\ttrain_loss: 1.5765\n",
      "Iteration: 56 of 277\ttrain_loss: 1.5715\n",
      "Iteration: 58 of 277\ttrain_loss: 1.4461\n",
      "Iteration: 60 of 277\ttrain_loss: 1.5687\n",
      "Iteration: 62 of 277\ttrain_loss: 1.5144\n",
      "Iteration: 64 of 277\ttrain_loss: 1.6288\n",
      "Iteration: 66 of 277\ttrain_loss: 1.4514\n",
      "Iteration: 68 of 277\ttrain_loss: 1.5116\n",
      "Iteration: 70 of 277\ttrain_loss: 1.7894\n",
      "Iteration: 72 of 277\ttrain_loss: 1.6224\n",
      "Iteration: 74 of 277\ttrain_loss: 1.5013\n",
      "Iteration: 76 of 277\ttrain_loss: 1.5769\n",
      "Iteration: 78 of 277\ttrain_loss: 1.5546\n",
      "Iteration: 80 of 277\ttrain_loss: 1.5118\n",
      "Iteration: 82 of 277\ttrain_loss: 1.6044\n",
      "Iteration: 84 of 277\ttrain_loss: 1.6262\n",
      "Iteration: 86 of 277\ttrain_loss: 1.3597\n",
      "Iteration: 88 of 277\ttrain_loss: 1.6712\n",
      "Iteration: 90 of 277\ttrain_loss: 1.7510\n",
      "Iteration: 92 of 277\ttrain_loss: 1.5500\n",
      "Iteration: 94 of 277\ttrain_loss: 1.6212\n",
      "Iteration: 96 of 277\ttrain_loss: 1.3559\n",
      "Iteration: 98 of 277\ttrain_loss: 1.6326\n",
      "Iteration: 100 of 277\ttrain_loss: 1.5802\n",
      "Iteration: 102 of 277\ttrain_loss: 1.4686\n",
      "Iteration: 104 of 277\ttrain_loss: 1.5870\n",
      "Iteration: 106 of 277\ttrain_loss: 1.7951\n",
      "Iteration: 108 of 277\ttrain_loss: 1.7338\n",
      "Iteration: 110 of 277\ttrain_loss: 1.5796\n",
      "Iteration: 112 of 277\ttrain_loss: 1.5201\n",
      "Iteration: 114 of 277\ttrain_loss: 1.6195\n",
      "Iteration: 116 of 277\ttrain_loss: 1.6864\n",
      "Iteration: 118 of 277\ttrain_loss: 1.6421\n",
      "Iteration: 120 of 277\ttrain_loss: 1.6174\n",
      "Iteration: 122 of 277\ttrain_loss: 1.5828\n",
      "Iteration: 124 of 277\ttrain_loss: 1.4435\n",
      "Iteration: 126 of 277\ttrain_loss: 1.6251\n",
      "Iteration: 128 of 277\ttrain_loss: 1.6207\n",
      "Iteration: 130 of 277\ttrain_loss: 1.5553\n",
      "Iteration: 132 of 277\ttrain_loss: 1.4725\n",
      "Iteration: 134 of 277\ttrain_loss: 1.5747\n",
      "Iteration: 136 of 277\ttrain_loss: 1.5746\n",
      "Iteration: 138 of 277\ttrain_loss: 1.5478\n",
      "Iteration: 140 of 277\ttrain_loss: 1.5356\n",
      "Iteration: 142 of 277\ttrain_loss: 1.5143\n",
      "Iteration: 144 of 277\ttrain_loss: 1.7300\n",
      "Iteration: 146 of 277\ttrain_loss: 1.4986\n",
      "Iteration: 148 of 277\ttrain_loss: 1.5140\n",
      "Iteration: 150 of 277\ttrain_loss: 1.6755\n",
      "Iteration: 152 of 277\ttrain_loss: 1.7847\n",
      "Iteration: 154 of 277\ttrain_loss: 1.5787\n",
      "Iteration: 156 of 277\ttrain_loss: 1.6494\n",
      "Iteration: 158 of 277\ttrain_loss: 1.5183\n",
      "Iteration: 160 of 277\ttrain_loss: 1.5238\n",
      "Iteration: 162 of 277\ttrain_loss: 1.4361\n",
      "Iteration: 164 of 277\ttrain_loss: 1.6388\n",
      "Iteration: 166 of 277\ttrain_loss: 1.4507\n",
      "Iteration: 168 of 277\ttrain_loss: 1.5986\n",
      "Iteration: 170 of 277\ttrain_loss: 1.5843\n",
      "Iteration: 172 of 277\ttrain_loss: 1.6417\n",
      "Iteration: 174 of 277\ttrain_loss: 1.7078\n",
      "Iteration: 176 of 277\ttrain_loss: 1.6147\n",
      "Iteration: 178 of 277\ttrain_loss: 1.6717\n",
      "Iteration: 180 of 277\ttrain_loss: 1.4486\n",
      "Iteration: 182 of 277\ttrain_loss: 1.6665\n",
      "Iteration: 184 of 277\ttrain_loss: 1.6636\n",
      "Iteration: 186 of 277\ttrain_loss: 1.6645\n",
      "Iteration: 188 of 277\ttrain_loss: 1.5847\n",
      "Iteration: 190 of 277\ttrain_loss: 1.5829\n",
      "Iteration: 192 of 277\ttrain_loss: 1.7279\n",
      "Iteration: 194 of 277\ttrain_loss: 1.6792\n",
      "Iteration: 196 of 277\ttrain_loss: 1.6626\n",
      "Iteration: 198 of 277\ttrain_loss: 1.4898\n",
      "Iteration: 200 of 277\ttrain_loss: 1.4923\n",
      "Iteration: 202 of 277\ttrain_loss: 1.7127\n",
      "Iteration: 204 of 277\ttrain_loss: 1.5787\n",
      "Iteration: 206 of 277\ttrain_loss: 1.4738\n",
      "Iteration: 208 of 277\ttrain_loss: 1.6807\n",
      "Iteration: 210 of 277\ttrain_loss: 1.7208\n",
      "Iteration: 212 of 277\ttrain_loss: 1.6532\n",
      "Iteration: 214 of 277\ttrain_loss: 1.5485\n",
      "Iteration: 216 of 277\ttrain_loss: 1.6660\n",
      "Iteration: 218 of 277\ttrain_loss: 1.4705\n",
      "Iteration: 220 of 277\ttrain_loss: 1.5451\n",
      "Iteration: 222 of 277\ttrain_loss: 1.6598\n",
      "Iteration: 224 of 277\ttrain_loss: 1.5121\n",
      "Iteration: 226 of 277\ttrain_loss: 1.5391\n",
      "Iteration: 228 of 277\ttrain_loss: 1.4262\n",
      "Iteration: 230 of 277\ttrain_loss: 1.6731\n",
      "Iteration: 232 of 277\ttrain_loss: 1.5264\n",
      "Iteration: 234 of 277\ttrain_loss: 1.7690\n",
      "Iteration: 236 of 277\ttrain_loss: 1.7781\n",
      "Iteration: 238 of 277\ttrain_loss: 1.4314\n",
      "Iteration: 240 of 277\ttrain_loss: 1.6090\n",
      "Iteration: 242 of 277\ttrain_loss: 1.6601\n",
      "Iteration: 244 of 277\ttrain_loss: 1.6004\n",
      "Iteration: 246 of 277\ttrain_loss: 1.6147\n",
      "Iteration: 248 of 277\ttrain_loss: 1.5604\n",
      "Iteration: 250 of 277\ttrain_loss: 1.6646\n",
      "Iteration: 252 of 277\ttrain_loss: 1.5129\n",
      "Iteration: 254 of 277\ttrain_loss: 1.4877\n",
      "Iteration: 256 of 277\ttrain_loss: 1.6908\n",
      "Iteration: 258 of 277\ttrain_loss: 1.7995\n",
      "Iteration: 260 of 277\ttrain_loss: 1.6109\n",
      "Iteration: 262 of 277\ttrain_loss: 1.6206\n",
      "Iteration: 264 of 277\ttrain_loss: 1.5135\n",
      "Iteration: 266 of 277\ttrain_loss: 1.7267\n",
      "Iteration: 268 of 277\ttrain_loss: 1.6259\n",
      "Iteration: 270 of 277\ttrain_loss: 1.6666\n",
      "Iteration: 272 of 277\ttrain_loss: 1.6566\n",
      "Iteration: 274 of 277\ttrain_loss: 1.6651\n",
      "Iteration: 276 of 277\ttrain_loss: 1.6029\n",
      "Iteration: 277 of 277\ttrain_loss: 1.4901\n",
      "Average Score for this Epoch: 1.6016755104064941\n",
      "Eval_loss: 2.5462646484375\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 16 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.5898\n",
      "Iteration: 2 of 277\ttrain_loss: 1.5831\n",
      "Iteration: 4 of 277\ttrain_loss: 1.6517\n",
      "Iteration: 6 of 277\ttrain_loss: 1.5615\n",
      "Iteration: 8 of 277\ttrain_loss: 1.6462\n",
      "Iteration: 10 of 277\ttrain_loss: 1.5789\n",
      "Iteration: 12 of 277\ttrain_loss: 1.5959\n",
      "Iteration: 14 of 277\ttrain_loss: 1.5019\n",
      "Iteration: 16 of 277\ttrain_loss: 1.6421\n",
      "Iteration: 18 of 277\ttrain_loss: 1.7502\n",
      "Iteration: 20 of 277\ttrain_loss: 1.6430\n",
      "Iteration: 22 of 277\ttrain_loss: 1.5779\n",
      "Iteration: 24 of 277\ttrain_loss: 1.5808\n",
      "Iteration: 26 of 277\ttrain_loss: 1.4793\n",
      "Iteration: 28 of 277\ttrain_loss: 1.5543\n",
      "Iteration: 30 of 277\ttrain_loss: 1.4966\n",
      "Iteration: 32 of 277\ttrain_loss: 1.4680\n",
      "Iteration: 34 of 277\ttrain_loss: 1.3516\n",
      "Iteration: 36 of 277\ttrain_loss: 1.6318\n",
      "Iteration: 38 of 277\ttrain_loss: 1.5240\n",
      "Iteration: 40 of 277\ttrain_loss: 1.5338\n",
      "Iteration: 42 of 277\ttrain_loss: 1.5975\n",
      "Iteration: 44 of 277\ttrain_loss: 1.7889\n",
      "Iteration: 46 of 277\ttrain_loss: 1.4940\n",
      "Iteration: 48 of 277\ttrain_loss: 1.6010\n",
      "Iteration: 50 of 277\ttrain_loss: 1.5719\n",
      "Iteration: 52 of 277\ttrain_loss: 1.6073\n",
      "Iteration: 54 of 277\ttrain_loss: 1.6791\n",
      "Iteration: 56 of 277\ttrain_loss: 1.7021\n",
      "Iteration: 58 of 277\ttrain_loss: 1.6701\n",
      "Iteration: 60 of 277\ttrain_loss: 1.7664\n",
      "Iteration: 62 of 277\ttrain_loss: 1.6831\n",
      "Iteration: 64 of 277\ttrain_loss: 1.6914\n",
      "Iteration: 66 of 277\ttrain_loss: 1.5231\n",
      "Iteration: 68 of 277\ttrain_loss: 1.5492\n",
      "Iteration: 70 of 277\ttrain_loss: 1.4951\n",
      "Iteration: 72 of 277\ttrain_loss: 1.5758\n",
      "Iteration: 74 of 277\ttrain_loss: 1.5210\n",
      "Iteration: 76 of 277\ttrain_loss: 1.5077\n",
      "Iteration: 78 of 277\ttrain_loss: 1.7167\n",
      "Iteration: 80 of 277\ttrain_loss: 1.6062\n",
      "Iteration: 82 of 277\ttrain_loss: 1.7420\n",
      "Iteration: 84 of 277\ttrain_loss: 1.6911\n",
      "Iteration: 86 of 277\ttrain_loss: 1.6003\n",
      "Iteration: 88 of 277\ttrain_loss: 1.6144\n",
      "Iteration: 90 of 277\ttrain_loss: 1.5962\n",
      "Iteration: 92 of 277\ttrain_loss: 1.6169\n",
      "Iteration: 94 of 277\ttrain_loss: 1.4450\n",
      "Iteration: 96 of 277\ttrain_loss: 1.4845\n",
      "Iteration: 98 of 277\ttrain_loss: 1.5124\n",
      "Iteration: 100 of 277\ttrain_loss: 1.7305\n",
      "Iteration: 102 of 277\ttrain_loss: 1.6587\n",
      "Iteration: 104 of 277\ttrain_loss: 1.6448\n",
      "Iteration: 106 of 277\ttrain_loss: 1.6793\n",
      "Iteration: 108 of 277\ttrain_loss: 1.5230\n",
      "Iteration: 110 of 277\ttrain_loss: 1.5573\n",
      "Iteration: 112 of 277\ttrain_loss: 1.3129\n",
      "Iteration: 114 of 277\ttrain_loss: 1.4884\n",
      "Iteration: 116 of 277\ttrain_loss: 1.6742\n",
      "Iteration: 118 of 277\ttrain_loss: 1.5963\n",
      "Iteration: 120 of 277\ttrain_loss: 1.7178\n",
      "Iteration: 122 of 277\ttrain_loss: 1.5578\n",
      "Iteration: 124 of 277\ttrain_loss: 1.8114\n",
      "Iteration: 126 of 277\ttrain_loss: 1.7803\n",
      "Iteration: 128 of 277\ttrain_loss: 1.6743\n",
      "Iteration: 130 of 277\ttrain_loss: 1.5040\n",
      "Iteration: 132 of 277\ttrain_loss: 1.4991\n",
      "Iteration: 134 of 277\ttrain_loss: 1.6146\n",
      "Iteration: 136 of 277\ttrain_loss: 1.5487\n",
      "Iteration: 138 of 277\ttrain_loss: 1.4777\n",
      "Iteration: 140 of 277\ttrain_loss: 1.6415\n",
      "Iteration: 142 of 277\ttrain_loss: 1.6386\n",
      "Iteration: 144 of 277\ttrain_loss: 1.6661\n",
      "Iteration: 146 of 277\ttrain_loss: 1.5226\n",
      "Iteration: 148 of 277\ttrain_loss: 1.6181\n",
      "Iteration: 150 of 277\ttrain_loss: 1.6499\n",
      "Iteration: 152 of 277\ttrain_loss: 1.6496\n",
      "Iteration: 154 of 277\ttrain_loss: 1.6043\n",
      "Iteration: 156 of 277\ttrain_loss: 1.6562\n",
      "Iteration: 158 of 277\ttrain_loss: 1.6956\n",
      "Iteration: 160 of 277\ttrain_loss: 1.5966\n",
      "Iteration: 162 of 277\ttrain_loss: 1.6398\n",
      "Iteration: 164 of 277\ttrain_loss: 1.5886\n",
      "Iteration: 166 of 277\ttrain_loss: 1.6205\n",
      "Iteration: 168 of 277\ttrain_loss: 1.8258\n",
      "Iteration: 170 of 277\ttrain_loss: 1.5972\n",
      "Iteration: 172 of 277\ttrain_loss: 1.7064\n",
      "Iteration: 174 of 277\ttrain_loss: 1.6977\n",
      "Iteration: 176 of 277\ttrain_loss: 1.6310\n",
      "Iteration: 178 of 277\ttrain_loss: 1.5961\n",
      "Iteration: 180 of 277\ttrain_loss: 1.4521\n",
      "Iteration: 182 of 277\ttrain_loss: 2.3369\n",
      "Iteration: 184 of 277\ttrain_loss: 1.7082\n",
      "Iteration: 186 of 277\ttrain_loss: 1.5475\n",
      "Iteration: 188 of 277\ttrain_loss: 1.7302\n",
      "Iteration: 190 of 277\ttrain_loss: 1.6864\n",
      "Iteration: 192 of 277\ttrain_loss: 1.6703\n",
      "Iteration: 194 of 277\ttrain_loss: 1.5958\n",
      "Iteration: 196 of 277\ttrain_loss: 1.6357\n",
      "Iteration: 198 of 277\ttrain_loss: 1.6295\n",
      "Iteration: 200 of 277\ttrain_loss: 1.7452\n",
      "Iteration: 202 of 277\ttrain_loss: 1.8318\n",
      "Iteration: 204 of 277\ttrain_loss: 1.5893\n",
      "Iteration: 206 of 277\ttrain_loss: 1.5918\n",
      "Iteration: 208 of 277\ttrain_loss: 1.6300\n",
      "Iteration: 210 of 277\ttrain_loss: 1.6776\n",
      "Iteration: 212 of 277\ttrain_loss: 1.5876\n",
      "Iteration: 214 of 277\ttrain_loss: 1.6148\n",
      "Iteration: 216 of 277\ttrain_loss: 1.7220\n",
      "Iteration: 218 of 277\ttrain_loss: 1.4771\n",
      "Iteration: 220 of 277\ttrain_loss: 1.5121\n",
      "Iteration: 222 of 277\ttrain_loss: 1.5215\n",
      "Iteration: 224 of 277\ttrain_loss: 1.4965\n",
      "Iteration: 226 of 277\ttrain_loss: 1.5601\n",
      "Iteration: 228 of 277\ttrain_loss: 1.6940\n",
      "Iteration: 230 of 277\ttrain_loss: 1.7800\n",
      "Iteration: 232 of 277\ttrain_loss: 1.4982\n",
      "Iteration: 234 of 277\ttrain_loss: 1.7267\n",
      "Iteration: 236 of 277\ttrain_loss: 1.6079\n",
      "Iteration: 238 of 277\ttrain_loss: 1.6441\n",
      "Iteration: 240 of 277\ttrain_loss: 1.7473\n",
      "Iteration: 242 of 277\ttrain_loss: 1.6603\n",
      "Iteration: 244 of 277\ttrain_loss: 1.7049\n",
      "Iteration: 246 of 277\ttrain_loss: 1.8543\n",
      "Iteration: 248 of 277\ttrain_loss: 1.8675\n",
      "Iteration: 250 of 277\ttrain_loss: 1.6574\n",
      "Iteration: 252 of 277\ttrain_loss: 1.7915\n",
      "Iteration: 254 of 277\ttrain_loss: 1.7478\n",
      "Iteration: 256 of 277\ttrain_loss: 1.6790\n",
      "Iteration: 258 of 277\ttrain_loss: 1.7631\n",
      "Iteration: 260 of 277\ttrain_loss: 1.6314\n",
      "Iteration: 262 of 277\ttrain_loss: 1.6970\n",
      "Iteration: 264 of 277\ttrain_loss: 1.7256\n",
      "Iteration: 266 of 277\ttrain_loss: 1.5698\n",
      "Iteration: 268 of 277\ttrain_loss: 1.7594\n",
      "Iteration: 270 of 277\ttrain_loss: 1.6552\n",
      "Iteration: 272 of 277\ttrain_loss: 1.6083\n",
      "Iteration: 274 of 277\ttrain_loss: 2.0142\n",
      "Iteration: 276 of 277\ttrain_loss: 1.6787\n",
      "Iteration: 277 of 277\ttrain_loss: 1.6418\n",
      "Average Score for this Epoch: 1.6297807693481445\n",
      "Eval_loss: 2.577733278274536\n",
      "\n",
      "-------------------- Epoch 17 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.6649\n",
      "Iteration: 2 of 277\ttrain_loss: 1.6289\n",
      "Iteration: 4 of 277\ttrain_loss: 1.5188\n",
      "Iteration: 6 of 277\ttrain_loss: 1.7994\n",
      "Iteration: 8 of 277\ttrain_loss: 1.4826\n",
      "Iteration: 10 of 277\ttrain_loss: 1.6279\n",
      "Iteration: 12 of 277\ttrain_loss: 1.7569\n",
      "Iteration: 14 of 277\ttrain_loss: 1.6115\n",
      "Iteration: 16 of 277\ttrain_loss: 1.4343\n",
      "Iteration: 18 of 277\ttrain_loss: 1.6963\n",
      "Iteration: 20 of 277\ttrain_loss: 1.5307\n",
      "Iteration: 22 of 277\ttrain_loss: 1.6269\n",
      "Iteration: 24 of 277\ttrain_loss: 1.3952\n",
      "Iteration: 26 of 277\ttrain_loss: 1.5998\n",
      "Iteration: 28 of 277\ttrain_loss: 1.5129\n",
      "Iteration: 30 of 277\ttrain_loss: 1.5090\n",
      "Iteration: 32 of 277\ttrain_loss: 1.4652\n",
      "Iteration: 34 of 277\ttrain_loss: 1.5052\n",
      "Iteration: 36 of 277\ttrain_loss: 1.8397\n",
      "Iteration: 38 of 277\ttrain_loss: 1.6453\n",
      "Iteration: 40 of 277\ttrain_loss: 1.4897\n",
      "Iteration: 42 of 277\ttrain_loss: 1.6064\n",
      "Iteration: 44 of 277\ttrain_loss: 1.4943\n",
      "Iteration: 46 of 277\ttrain_loss: 1.5601\n",
      "Iteration: 48 of 277\ttrain_loss: 1.6299\n",
      "Iteration: 50 of 277\ttrain_loss: 1.5471\n",
      "Iteration: 52 of 277\ttrain_loss: 1.6215\n",
      "Iteration: 54 of 277\ttrain_loss: 1.7988\n",
      "Iteration: 56 of 277\ttrain_loss: 1.4458\n",
      "Iteration: 58 of 277\ttrain_loss: 1.6036\n",
      "Iteration: 60 of 277\ttrain_loss: 1.5925\n",
      "Iteration: 62 of 277\ttrain_loss: 1.7631\n",
      "Iteration: 64 of 277\ttrain_loss: 1.8627\n",
      "Iteration: 66 of 277\ttrain_loss: 1.5261\n",
      "Iteration: 68 of 277\ttrain_loss: 1.5213\n",
      "Iteration: 70 of 277\ttrain_loss: 1.6034\n",
      "Iteration: 72 of 277\ttrain_loss: 1.7027\n",
      "Iteration: 74 of 277\ttrain_loss: 1.4135\n",
      "Iteration: 76 of 277\ttrain_loss: 1.7737\n",
      "Iteration: 78 of 277\ttrain_loss: 1.5485\n",
      "Iteration: 80 of 277\ttrain_loss: 1.5518\n",
      "Iteration: 82 of 277\ttrain_loss: 1.4288\n",
      "Iteration: 84 of 277\ttrain_loss: 1.5528\n",
      "Iteration: 86 of 277\ttrain_loss: 1.7215\n",
      "Iteration: 88 of 277\ttrain_loss: 1.6707\n",
      "Iteration: 90 of 277\ttrain_loss: 1.4972\n",
      "Iteration: 92 of 277\ttrain_loss: 1.6390\n",
      "Iteration: 94 of 277\ttrain_loss: 1.5295\n",
      "Iteration: 96 of 277\ttrain_loss: 1.5730\n",
      "Iteration: 98 of 277\ttrain_loss: 1.5711\n",
      "Iteration: 100 of 277\ttrain_loss: 1.7888\n",
      "Iteration: 102 of 277\ttrain_loss: 1.5222\n",
      "Iteration: 104 of 277\ttrain_loss: 1.6449\n",
      "Iteration: 106 of 277\ttrain_loss: 1.5047\n",
      "Iteration: 108 of 277\ttrain_loss: 1.7548\n",
      "Iteration: 110 of 277\ttrain_loss: 1.7155\n",
      "Iteration: 112 of 277\ttrain_loss: 1.6136\n",
      "Iteration: 114 of 277\ttrain_loss: 1.5886\n",
      "Iteration: 116 of 277\ttrain_loss: 1.6407\n",
      "Iteration: 118 of 277\ttrain_loss: 1.5833\n",
      "Iteration: 120 of 277\ttrain_loss: 1.6581\n",
      "Iteration: 122 of 277\ttrain_loss: 1.6162\n",
      "Iteration: 124 of 277\ttrain_loss: 1.6478\n",
      "Iteration: 126 of 277\ttrain_loss: 1.7597\n",
      "Iteration: 128 of 277\ttrain_loss: 1.5351\n",
      "Iteration: 130 of 277\ttrain_loss: 1.4693\n",
      "Iteration: 132 of 277\ttrain_loss: 1.6156\n",
      "Iteration: 134 of 277\ttrain_loss: 1.5741\n",
      "Iteration: 136 of 277\ttrain_loss: 1.5779\n",
      "Iteration: 138 of 277\ttrain_loss: 1.6628\n",
      "Iteration: 140 of 277\ttrain_loss: 1.5054\n",
      "Iteration: 142 of 277\ttrain_loss: 1.7641\n",
      "Iteration: 144 of 277\ttrain_loss: 1.6643\n",
      "Iteration: 146 of 277\ttrain_loss: 1.7981\n",
      "Iteration: 148 of 277\ttrain_loss: 1.6683\n",
      "Iteration: 150 of 277\ttrain_loss: 1.7015\n",
      "Iteration: 152 of 277\ttrain_loss: 1.7585\n",
      "Iteration: 154 of 277\ttrain_loss: 1.7558\n",
      "Iteration: 156 of 277\ttrain_loss: 1.7211\n",
      "Iteration: 158 of 277\ttrain_loss: 1.6120\n",
      "Iteration: 160 of 277\ttrain_loss: 1.8479\n",
      "Iteration: 162 of 277\ttrain_loss: 1.8601\n",
      "Iteration: 164 of 277\ttrain_loss: 1.5567\n",
      "Iteration: 166 of 277\ttrain_loss: 1.7304\n",
      "Iteration: 168 of 277\ttrain_loss: 1.4923\n",
      "Iteration: 170 of 277\ttrain_loss: 1.5780\n",
      "Iteration: 172 of 277\ttrain_loss: 1.8456\n",
      "Iteration: 174 of 277\ttrain_loss: 1.9167\n",
      "Iteration: 176 of 277\ttrain_loss: 1.6414\n",
      "Iteration: 178 of 277\ttrain_loss: 1.6753\n",
      "Iteration: 180 of 277\ttrain_loss: 1.8716\n",
      "Iteration: 182 of 277\ttrain_loss: 1.9143\n",
      "Iteration: 184 of 277\ttrain_loss: 1.6125\n",
      "Iteration: 186 of 277\ttrain_loss: 1.4797\n",
      "Iteration: 188 of 277\ttrain_loss: 1.5980\n",
      "Iteration: 190 of 277\ttrain_loss: 1.7402\n",
      "Iteration: 192 of 277\ttrain_loss: 1.7997\n",
      "Iteration: 194 of 277\ttrain_loss: 1.7716\n",
      "Iteration: 196 of 277\ttrain_loss: 1.7221\n",
      "Iteration: 198 of 277\ttrain_loss: 1.5712\n",
      "Iteration: 200 of 277\ttrain_loss: 1.5719\n",
      "Iteration: 202 of 277\ttrain_loss: 1.7568\n",
      "Iteration: 204 of 277\ttrain_loss: 1.5797\n",
      "Iteration: 206 of 277\ttrain_loss: 1.7701\n",
      "Iteration: 208 of 277\ttrain_loss: 1.6403\n",
      "Iteration: 210 of 277\ttrain_loss: 1.7916\n",
      "Iteration: 212 of 277\ttrain_loss: 1.8520\n",
      "Iteration: 214 of 277\ttrain_loss: 1.6347\n",
      "Iteration: 216 of 277\ttrain_loss: 1.6773\n",
      "Iteration: 218 of 277\ttrain_loss: 1.5574\n",
      "Iteration: 220 of 277\ttrain_loss: 1.8145\n",
      "Iteration: 222 of 277\ttrain_loss: 1.6603\n",
      "Iteration: 224 of 277\ttrain_loss: 1.8147\n",
      "Iteration: 226 of 277\ttrain_loss: 1.7103\n",
      "Iteration: 228 of 277\ttrain_loss: 1.7131\n",
      "Iteration: 230 of 277\ttrain_loss: 1.5519\n",
      "Iteration: 232 of 277\ttrain_loss: 1.7065\n",
      "Iteration: 234 of 277\ttrain_loss: 1.7747\n",
      "Iteration: 236 of 277\ttrain_loss: 1.7013\n",
      "Iteration: 238 of 277\ttrain_loss: 1.9820\n",
      "Iteration: 240 of 277\ttrain_loss: 1.6592\n",
      "Iteration: 242 of 277\ttrain_loss: 1.9534\n",
      "Iteration: 244 of 277\ttrain_loss: 1.6770\n",
      "Iteration: 246 of 277\ttrain_loss: 1.5428\n",
      "Iteration: 248 of 277\ttrain_loss: 1.8265\n",
      "Iteration: 250 of 277\ttrain_loss: 1.4655\n",
      "Iteration: 252 of 277\ttrain_loss: 1.7168\n",
      "Iteration: 254 of 277\ttrain_loss: 1.8177\n",
      "Iteration: 256 of 277\ttrain_loss: 1.7478\n",
      "Iteration: 258 of 277\ttrain_loss: 1.6375\n",
      "Iteration: 260 of 277\ttrain_loss: 1.6564\n",
      "Iteration: 262 of 277\ttrain_loss: 1.6789\n",
      "Iteration: 264 of 277\ttrain_loss: 1.8526\n",
      "Iteration: 266 of 277\ttrain_loss: 1.6118\n",
      "Iteration: 268 of 277\ttrain_loss: 1.6420\n",
      "Iteration: 270 of 277\ttrain_loss: 1.8788\n",
      "Iteration: 272 of 277\ttrain_loss: 1.6542\n",
      "Iteration: 274 of 277\ttrain_loss: 1.8019\n",
      "Iteration: 276 of 277\ttrain_loss: 1.7050\n",
      "Iteration: 277 of 277\ttrain_loss: 1.7581\n",
      "Average Score for this Epoch: 1.6539968252182007\n",
      "Eval_loss: 2.573861837387085\n",
      "\n",
      "-------------------- Epoch 18 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.4715\n",
      "Iteration: 2 of 277\ttrain_loss: 1.4930\n",
      "Iteration: 4 of 277\ttrain_loss: 1.4556\n",
      "Iteration: 6 of 277\ttrain_loss: 1.6136\n",
      "Iteration: 8 of 277\ttrain_loss: 1.5125\n",
      "Iteration: 10 of 277\ttrain_loss: 1.5892\n",
      "Iteration: 12 of 277\ttrain_loss: 1.5238\n",
      "Iteration: 14 of 277\ttrain_loss: 1.5727\n",
      "Iteration: 16 of 277\ttrain_loss: 1.6135\n",
      "Iteration: 18 of 277\ttrain_loss: 1.5507\n",
      "Iteration: 20 of 277\ttrain_loss: 1.5719\n",
      "Iteration: 22 of 277\ttrain_loss: 1.6591\n",
      "Iteration: 24 of 277\ttrain_loss: 1.5108\n",
      "Iteration: 26 of 277\ttrain_loss: 1.4965\n",
      "Iteration: 28 of 277\ttrain_loss: 1.6089\n",
      "Iteration: 30 of 277\ttrain_loss: 1.5290\n",
      "Iteration: 32 of 277\ttrain_loss: 1.4947\n",
      "Iteration: 34 of 277\ttrain_loss: 1.5529\n",
      "Iteration: 36 of 277\ttrain_loss: 1.7528\n",
      "Iteration: 38 of 277\ttrain_loss: 1.5698\n",
      "Iteration: 40 of 277\ttrain_loss: 1.5830\n",
      "Iteration: 42 of 277\ttrain_loss: 1.4614\n",
      "Iteration: 44 of 277\ttrain_loss: 1.4262\n",
      "Iteration: 46 of 277\ttrain_loss: 1.5225\n",
      "Iteration: 48 of 277\ttrain_loss: 1.7270\n",
      "Iteration: 50 of 277\ttrain_loss: 1.6092\n",
      "Iteration: 52 of 277\ttrain_loss: 1.6481\n",
      "Iteration: 54 of 277\ttrain_loss: 1.5671\n",
      "Iteration: 56 of 277\ttrain_loss: 1.5823\n",
      "Iteration: 58 of 277\ttrain_loss: 1.6173\n",
      "Iteration: 60 of 277\ttrain_loss: 1.5375\n",
      "Iteration: 62 of 277\ttrain_loss: 1.6112\n",
      "Iteration: 64 of 277\ttrain_loss: 1.4595\n",
      "Iteration: 66 of 277\ttrain_loss: 1.4350\n",
      "Iteration: 68 of 277\ttrain_loss: 1.5286\n",
      "Iteration: 70 of 277\ttrain_loss: 1.7221\n",
      "Iteration: 72 of 277\ttrain_loss: 1.4493\n",
      "Iteration: 74 of 277\ttrain_loss: 1.5728\n",
      "Iteration: 76 of 277\ttrain_loss: 1.4983\n",
      "Iteration: 78 of 277\ttrain_loss: 1.6133\n",
      "Iteration: 80 of 277\ttrain_loss: 1.5423\n",
      "Iteration: 82 of 277\ttrain_loss: 1.4981\n",
      "Iteration: 84 of 277\ttrain_loss: 1.5759\n",
      "Iteration: 86 of 277\ttrain_loss: 1.6604\n",
      "Iteration: 88 of 277\ttrain_loss: 1.4749\n",
      "Iteration: 90 of 277\ttrain_loss: 1.6633\n",
      "Iteration: 92 of 277\ttrain_loss: 1.5733\n",
      "Iteration: 94 of 277\ttrain_loss: 1.5778\n",
      "Iteration: 96 of 277\ttrain_loss: 1.5878\n",
      "Iteration: 98 of 277\ttrain_loss: 1.5816\n",
      "Iteration: 100 of 277\ttrain_loss: 1.5265\n",
      "Iteration: 102 of 277\ttrain_loss: 1.7090\n",
      "Iteration: 104 of 277\ttrain_loss: 1.5733\n",
      "Iteration: 106 of 277\ttrain_loss: 1.5481\n",
      "Iteration: 108 of 277\ttrain_loss: 1.6438\n",
      "Iteration: 110 of 277\ttrain_loss: 1.6693\n",
      "Iteration: 112 of 277\ttrain_loss: 1.6140\n",
      "Iteration: 114 of 277\ttrain_loss: 1.5411\n",
      "Iteration: 116 of 277\ttrain_loss: 1.5330\n",
      "Iteration: 118 of 277\ttrain_loss: 1.6162\n",
      "Iteration: 120 of 277\ttrain_loss: 1.4732\n",
      "Iteration: 122 of 277\ttrain_loss: 1.5988\n",
      "Iteration: 124 of 277\ttrain_loss: 1.4723\n",
      "Iteration: 126 of 277\ttrain_loss: 1.5769\n",
      "Iteration: 128 of 277\ttrain_loss: 1.4847\n",
      "Iteration: 130 of 277\ttrain_loss: 1.8277\n",
      "Iteration: 132 of 277\ttrain_loss: 1.5359\n",
      "Iteration: 134 of 277\ttrain_loss: 1.4703\n",
      "Iteration: 136 of 277\ttrain_loss: 1.5558\n",
      "Iteration: 138 of 277\ttrain_loss: 1.6330\n",
      "Iteration: 140 of 277\ttrain_loss: 1.5585\n",
      "Iteration: 142 of 277\ttrain_loss: 1.5090\n",
      "Iteration: 144 of 277\ttrain_loss: 1.5623\n",
      "Iteration: 146 of 277\ttrain_loss: 1.7053\n",
      "Iteration: 148 of 277\ttrain_loss: 1.5420\n",
      "Iteration: 150 of 277\ttrain_loss: 1.5479\n",
      "Iteration: 152 of 277\ttrain_loss: 1.5768\n",
      "Iteration: 154 of 277\ttrain_loss: 1.6849\n",
      "Iteration: 156 of 277\ttrain_loss: 1.5933\n",
      "Iteration: 158 of 277\ttrain_loss: 1.5511\n",
      "Iteration: 160 of 277\ttrain_loss: 1.5375\n",
      "Iteration: 162 of 277\ttrain_loss: 1.5171\n",
      "Iteration: 164 of 277\ttrain_loss: 1.5031\n",
      "Iteration: 166 of 277\ttrain_loss: 1.4256\n",
      "Iteration: 168 of 277\ttrain_loss: 1.5640\n",
      "Iteration: 170 of 277\ttrain_loss: 1.6829\n",
      "Iteration: 172 of 277\ttrain_loss: 1.5433\n",
      "Iteration: 174 of 277\ttrain_loss: 1.6785\n",
      "Iteration: 176 of 277\ttrain_loss: 1.5137\n",
      "Iteration: 178 of 277\ttrain_loss: 1.4587\n",
      "Iteration: 180 of 277\ttrain_loss: 1.5012\n",
      "Iteration: 182 of 277\ttrain_loss: 1.5385\n",
      "Iteration: 184 of 277\ttrain_loss: 1.5979\n",
      "Iteration: 186 of 277\ttrain_loss: 1.9037\n",
      "Iteration: 188 of 277\ttrain_loss: 1.7247\n",
      "Iteration: 190 of 277\ttrain_loss: 1.6925\n",
      "Iteration: 192 of 277\ttrain_loss: 1.5636\n",
      "Iteration: 194 of 277\ttrain_loss: 1.6142\n",
      "Iteration: 196 of 277\ttrain_loss: 1.4265\n",
      "Iteration: 198 of 277\ttrain_loss: 1.6007\n",
      "Iteration: 200 of 277\ttrain_loss: 1.4790\n",
      "Iteration: 202 of 277\ttrain_loss: 1.5471\n",
      "Iteration: 204 of 277\ttrain_loss: 1.5351\n",
      "Iteration: 206 of 277\ttrain_loss: 1.6645\n",
      "Iteration: 208 of 277\ttrain_loss: 1.4000\n",
      "Iteration: 210 of 277\ttrain_loss: 1.3654\n",
      "Iteration: 212 of 277\ttrain_loss: 1.4480\n",
      "Iteration: 214 of 277\ttrain_loss: 1.6146\n",
      "Iteration: 216 of 277\ttrain_loss: 1.5459\n",
      "Iteration: 218 of 277\ttrain_loss: 1.5087\n",
      "Iteration: 220 of 277\ttrain_loss: 1.4425\n",
      "Iteration: 222 of 277\ttrain_loss: 1.4071\n",
      "Iteration: 224 of 277\ttrain_loss: 1.5646\n",
      "Iteration: 226 of 277\ttrain_loss: 1.5163\n",
      "Iteration: 228 of 277\ttrain_loss: 1.6334\n",
      "Iteration: 230 of 277\ttrain_loss: 1.3940\n",
      "Iteration: 232 of 277\ttrain_loss: 1.6825\n",
      "Iteration: 234 of 277\ttrain_loss: 1.6070\n",
      "Iteration: 236 of 277\ttrain_loss: 1.3832\n",
      "Iteration: 238 of 277\ttrain_loss: 1.5007\n",
      "Iteration: 240 of 277\ttrain_loss: 1.4969\n",
      "Iteration: 242 of 277\ttrain_loss: 1.6344\n",
      "Iteration: 244 of 277\ttrain_loss: 1.6894\n",
      "Iteration: 246 of 277\ttrain_loss: 1.4602\n",
      "Iteration: 248 of 277\ttrain_loss: 1.6523\n",
      "Iteration: 250 of 277\ttrain_loss: 1.6238\n",
      "Iteration: 252 of 277\ttrain_loss: 1.5330\n",
      "Iteration: 254 of 277\ttrain_loss: 1.4430\n",
      "Iteration: 256 of 277\ttrain_loss: 1.5992\n",
      "Iteration: 258 of 277\ttrain_loss: 1.5363\n",
      "Iteration: 260 of 277\ttrain_loss: 1.7041\n",
      "Iteration: 262 of 277\ttrain_loss: 1.5236\n",
      "Iteration: 264 of 277\ttrain_loss: 1.6608\n",
      "Iteration: 266 of 277\ttrain_loss: 1.7017\n",
      "Iteration: 268 of 277\ttrain_loss: 1.4006\n",
      "Iteration: 270 of 277\ttrain_loss: 1.4082\n",
      "Iteration: 272 of 277\ttrain_loss: 1.6141\n",
      "Iteration: 274 of 277\ttrain_loss: 1.5330\n",
      "Iteration: 276 of 277\ttrain_loss: 1.5208\n",
      "Iteration: 277 of 277\ttrain_loss: 1.4535\n",
      "Average Score for this Epoch: 1.5677576065063477\n",
      "Eval_loss: 2.5514068603515625\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 19 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.4670\n",
      "Iteration: 2 of 277\ttrain_loss: 1.4024\n",
      "Iteration: 4 of 277\ttrain_loss: 1.4388\n",
      "Iteration: 6 of 277\ttrain_loss: 1.3139\n",
      "Iteration: 8 of 277\ttrain_loss: 1.3590\n",
      "Iteration: 10 of 277\ttrain_loss: 1.2395\n",
      "Iteration: 12 of 277\ttrain_loss: 1.4505\n",
      "Iteration: 14 of 277\ttrain_loss: 1.3397\n",
      "Iteration: 16 of 277\ttrain_loss: 1.2422\n",
      "Iteration: 18 of 277\ttrain_loss: 1.4486\n",
      "Iteration: 20 of 277\ttrain_loss: 1.4482\n",
      "Iteration: 22 of 277\ttrain_loss: 1.2508\n",
      "Iteration: 24 of 277\ttrain_loss: 1.4664\n",
      "Iteration: 26 of 277\ttrain_loss: 1.3618\n",
      "Iteration: 28 of 277\ttrain_loss: 1.5200\n",
      "Iteration: 30 of 277\ttrain_loss: 1.4148\n",
      "Iteration: 32 of 277\ttrain_loss: 1.4241\n",
      "Iteration: 34 of 277\ttrain_loss: 1.3350\n",
      "Iteration: 36 of 277\ttrain_loss: 1.4317\n",
      "Iteration: 38 of 277\ttrain_loss: 1.3652\n",
      "Iteration: 40 of 277\ttrain_loss: 1.3945\n",
      "Iteration: 42 of 277\ttrain_loss: 1.3716\n",
      "Iteration: 44 of 277\ttrain_loss: 1.3902\n",
      "Iteration: 46 of 277\ttrain_loss: 1.4389\n",
      "Iteration: 48 of 277\ttrain_loss: 1.5857\n",
      "Iteration: 50 of 277\ttrain_loss: 1.4634\n",
      "Iteration: 52 of 277\ttrain_loss: 1.3321\n",
      "Iteration: 54 of 277\ttrain_loss: 1.2814\n",
      "Iteration: 56 of 277\ttrain_loss: 1.4622\n",
      "Iteration: 58 of 277\ttrain_loss: 1.3999\n",
      "Iteration: 60 of 277\ttrain_loss: 1.6981\n",
      "Iteration: 62 of 277\ttrain_loss: 1.4226\n",
      "Iteration: 64 of 277\ttrain_loss: 1.3854\n",
      "Iteration: 66 of 277\ttrain_loss: 1.4313\n",
      "Iteration: 68 of 277\ttrain_loss: 1.3688\n",
      "Iteration: 70 of 277\ttrain_loss: 1.3651\n",
      "Iteration: 72 of 277\ttrain_loss: 1.3757\n",
      "Iteration: 74 of 277\ttrain_loss: 1.4633\n",
      "Iteration: 76 of 277\ttrain_loss: 1.4583\n",
      "Iteration: 78 of 277\ttrain_loss: 1.4240\n",
      "Iteration: 80 of 277\ttrain_loss: 1.3352\n",
      "Iteration: 82 of 277\ttrain_loss: 1.5374\n",
      "Iteration: 84 of 277\ttrain_loss: 1.3790\n",
      "Iteration: 86 of 277\ttrain_loss: 1.3656\n",
      "Iteration: 88 of 277\ttrain_loss: 1.6727\n",
      "Iteration: 90 of 277\ttrain_loss: 1.3867\n",
      "Iteration: 92 of 277\ttrain_loss: 1.3742\n",
      "Iteration: 94 of 277\ttrain_loss: 1.6008\n",
      "Iteration: 96 of 277\ttrain_loss: 1.3277\n",
      "Iteration: 98 of 277\ttrain_loss: 1.4908\n",
      "Iteration: 100 of 277\ttrain_loss: 1.4042\n",
      "Iteration: 102 of 277\ttrain_loss: 1.3872\n",
      "Iteration: 104 of 277\ttrain_loss: 1.3911\n",
      "Iteration: 106 of 277\ttrain_loss: 1.4344\n",
      "Iteration: 108 of 277\ttrain_loss: 1.5115\n",
      "Iteration: 110 of 277\ttrain_loss: 1.5331\n",
      "Iteration: 112 of 277\ttrain_loss: 1.2889\n",
      "Iteration: 114 of 277\ttrain_loss: 1.4879\n",
      "Iteration: 116 of 277\ttrain_loss: 1.3521\n",
      "Iteration: 118 of 277\ttrain_loss: 1.3447\n",
      "Iteration: 120 of 277\ttrain_loss: 1.4811\n",
      "Iteration: 122 of 277\ttrain_loss: 1.3534\n",
      "Iteration: 124 of 277\ttrain_loss: 1.3372\n",
      "Iteration: 126 of 277\ttrain_loss: 1.5224\n",
      "Iteration: 128 of 277\ttrain_loss: 1.2370\n",
      "Iteration: 130 of 277\ttrain_loss: 1.5160\n",
      "Iteration: 132 of 277\ttrain_loss: 1.5466\n",
      "Iteration: 134 of 277\ttrain_loss: 1.5161\n",
      "Iteration: 136 of 277\ttrain_loss: 1.4545\n",
      "Iteration: 138 of 277\ttrain_loss: 1.4088\n",
      "Iteration: 140 of 277\ttrain_loss: 1.4454\n",
      "Iteration: 142 of 277\ttrain_loss: 1.3878\n",
      "Iteration: 144 of 277\ttrain_loss: 1.3295\n",
      "Iteration: 146 of 277\ttrain_loss: 1.3892\n",
      "Iteration: 148 of 277\ttrain_loss: 1.4768\n",
      "Iteration: 150 of 277\ttrain_loss: 1.4332\n",
      "Iteration: 152 of 277\ttrain_loss: 1.7250\n",
      "Iteration: 154 of 277\ttrain_loss: 1.4338\n",
      "Iteration: 156 of 277\ttrain_loss: 1.4813\n",
      "Iteration: 158 of 277\ttrain_loss: 1.2822\n",
      "Iteration: 160 of 277\ttrain_loss: 1.4508\n",
      "Iteration: 162 of 277\ttrain_loss: 1.5009\n",
      "Iteration: 164 of 277\ttrain_loss: 1.3235\n",
      "Iteration: 166 of 277\ttrain_loss: 1.3415\n",
      "Iteration: 168 of 277\ttrain_loss: 1.5111\n",
      "Iteration: 170 of 277\ttrain_loss: 1.5154\n",
      "Iteration: 172 of 277\ttrain_loss: 1.3050\n",
      "Iteration: 174 of 277\ttrain_loss: 1.5149\n",
      "Iteration: 176 of 277\ttrain_loss: 1.4379\n",
      "Iteration: 178 of 277\ttrain_loss: 1.2295\n",
      "Iteration: 180 of 277\ttrain_loss: 1.3981\n",
      "Iteration: 182 of 277\ttrain_loss: 1.5137\n",
      "Iteration: 184 of 277\ttrain_loss: 1.4216\n",
      "Iteration: 186 of 277\ttrain_loss: 1.5479\n",
      "Iteration: 188 of 277\ttrain_loss: 1.3526\n",
      "Iteration: 190 of 277\ttrain_loss: 1.5973\n",
      "Iteration: 192 of 277\ttrain_loss: 1.3813\n",
      "Iteration: 194 of 277\ttrain_loss: 1.4417\n",
      "Iteration: 196 of 277\ttrain_loss: 1.5113\n",
      "Iteration: 198 of 277\ttrain_loss: 1.5526\n",
      "Iteration: 200 of 277\ttrain_loss: 1.4407\n",
      "Iteration: 202 of 277\ttrain_loss: 1.4458\n",
      "Iteration: 204 of 277\ttrain_loss: 1.5347\n",
      "Iteration: 206 of 277\ttrain_loss: 1.4361\n",
      "Iteration: 208 of 277\ttrain_loss: 1.4143\n",
      "Iteration: 210 of 277\ttrain_loss: 1.3922\n",
      "Iteration: 212 of 277\ttrain_loss: 1.6248\n",
      "Iteration: 214 of 277\ttrain_loss: 1.2612\n",
      "Iteration: 216 of 277\ttrain_loss: 1.5568\n",
      "Iteration: 218 of 277\ttrain_loss: 1.4098\n",
      "Iteration: 220 of 277\ttrain_loss: 1.4003\n",
      "Iteration: 222 of 277\ttrain_loss: 1.4266\n",
      "Iteration: 224 of 277\ttrain_loss: 1.3765\n",
      "Iteration: 226 of 277\ttrain_loss: 1.3203\n",
      "Iteration: 228 of 277\ttrain_loss: 1.4470\n",
      "Iteration: 230 of 277\ttrain_loss: 1.4537\n",
      "Iteration: 232 of 277\ttrain_loss: 1.3995\n",
      "Iteration: 234 of 277\ttrain_loss: 1.4393\n",
      "Iteration: 236 of 277\ttrain_loss: 1.4613\n",
      "Iteration: 238 of 277\ttrain_loss: 1.5451\n",
      "Iteration: 240 of 277\ttrain_loss: 1.4284\n",
      "Iteration: 242 of 277\ttrain_loss: 1.5515\n",
      "Iteration: 244 of 277\ttrain_loss: 1.3185\n",
      "Iteration: 246 of 277\ttrain_loss: 1.4790\n",
      "Iteration: 248 of 277\ttrain_loss: 1.3745\n",
      "Iteration: 250 of 277\ttrain_loss: 1.3008\n",
      "Iteration: 252 of 277\ttrain_loss: 1.3950\n",
      "Iteration: 254 of 277\ttrain_loss: 1.2828\n",
      "Iteration: 256 of 277\ttrain_loss: 1.5377\n",
      "Iteration: 258 of 277\ttrain_loss: 1.4726\n",
      "Iteration: 260 of 277\ttrain_loss: 1.4093\n",
      "Iteration: 262 of 277\ttrain_loss: 1.4973\n",
      "Iteration: 264 of 277\ttrain_loss: 1.6024\n",
      "Iteration: 266 of 277\ttrain_loss: 1.5941\n",
      "Iteration: 268 of 277\ttrain_loss: 1.4328\n",
      "Iteration: 270 of 277\ttrain_loss: 1.4204\n",
      "Iteration: 272 of 277\ttrain_loss: 1.4312\n",
      "Iteration: 274 of 277\ttrain_loss: 1.4956\n",
      "Iteration: 276 of 277\ttrain_loss: 1.3668\n",
      "Iteration: 277 of 277\ttrain_loss: 1.4481\n",
      "Average Score for this Epoch: 1.4309580326080322\n",
      "Eval_loss: 2.502906084060669\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 20 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.3256\n",
      "Iteration: 2 of 277\ttrain_loss: 1.3852\n",
      "Iteration: 4 of 277\ttrain_loss: 1.3194\n",
      "Iteration: 6 of 277\ttrain_loss: 1.2053\n",
      "Iteration: 8 of 277\ttrain_loss: 1.3527\n",
      "Iteration: 10 of 277\ttrain_loss: 1.1772\n",
      "Iteration: 12 of 277\ttrain_loss: 1.2901\n",
      "Iteration: 14 of 277\ttrain_loss: 1.4350\n",
      "Iteration: 16 of 277\ttrain_loss: 1.2527\n",
      "Iteration: 18 of 277\ttrain_loss: 1.2691\n",
      "Iteration: 20 of 277\ttrain_loss: 1.2023\n",
      "Iteration: 22 of 277\ttrain_loss: 1.2816\n",
      "Iteration: 24 of 277\ttrain_loss: 1.2114\n",
      "Iteration: 26 of 277\ttrain_loss: 1.2844\n",
      "Iteration: 28 of 277\ttrain_loss: 1.3166\n",
      "Iteration: 30 of 277\ttrain_loss: 1.1825\n",
      "Iteration: 32 of 277\ttrain_loss: 1.3270\n",
      "Iteration: 34 of 277\ttrain_loss: 1.3177\n",
      "Iteration: 36 of 277\ttrain_loss: 1.3310\n",
      "Iteration: 38 of 277\ttrain_loss: 1.3330\n",
      "Iteration: 40 of 277\ttrain_loss: 1.3831\n",
      "Iteration: 42 of 277\ttrain_loss: 1.5301\n",
      "Iteration: 44 of 277\ttrain_loss: 1.4182\n",
      "Iteration: 46 of 277\ttrain_loss: 1.3013\n",
      "Iteration: 48 of 277\ttrain_loss: 1.2207\n",
      "Iteration: 50 of 277\ttrain_loss: 1.3686\n",
      "Iteration: 52 of 277\ttrain_loss: 1.2090\n",
      "Iteration: 54 of 277\ttrain_loss: 1.2197\n",
      "Iteration: 56 of 277\ttrain_loss: 1.3679\n",
      "Iteration: 58 of 277\ttrain_loss: 1.3023\n",
      "Iteration: 60 of 277\ttrain_loss: 1.4254\n",
      "Iteration: 62 of 277\ttrain_loss: 1.4777\n",
      "Iteration: 64 of 277\ttrain_loss: 1.3048\n",
      "Iteration: 66 of 277\ttrain_loss: 1.3444\n",
      "Iteration: 68 of 277\ttrain_loss: 1.1913\n",
      "Iteration: 70 of 277\ttrain_loss: 1.3005\n",
      "Iteration: 72 of 277\ttrain_loss: 1.3637\n",
      "Iteration: 74 of 277\ttrain_loss: 1.3900\n",
      "Iteration: 76 of 277\ttrain_loss: 1.3920\n",
      "Iteration: 78 of 277\ttrain_loss: 1.4012\n",
      "Iteration: 80 of 277\ttrain_loss: 1.2569\n",
      "Iteration: 82 of 277\ttrain_loss: 1.2499\n",
      "Iteration: 84 of 277\ttrain_loss: 1.3467\n",
      "Iteration: 86 of 277\ttrain_loss: 1.3366\n",
      "Iteration: 88 of 277\ttrain_loss: 1.2524\n",
      "Iteration: 90 of 277\ttrain_loss: 1.3139\n",
      "Iteration: 92 of 277\ttrain_loss: 1.3100\n",
      "Iteration: 94 of 277\ttrain_loss: 1.3689\n",
      "Iteration: 96 of 277\ttrain_loss: 1.3087\n",
      "Iteration: 98 of 277\ttrain_loss: 1.4285\n",
      "Iteration: 100 of 277\ttrain_loss: 1.2913\n",
      "Iteration: 102 of 277\ttrain_loss: 1.3007\n",
      "Iteration: 104 of 277\ttrain_loss: 1.3063\n",
      "Iteration: 106 of 277\ttrain_loss: 1.2669\n",
      "Iteration: 108 of 277\ttrain_loss: 1.3471\n",
      "Iteration: 110 of 277\ttrain_loss: 1.3654\n",
      "Iteration: 112 of 277\ttrain_loss: 1.2801\n",
      "Iteration: 114 of 277\ttrain_loss: 1.4881\n",
      "Iteration: 116 of 277\ttrain_loss: 1.3694\n",
      "Iteration: 118 of 277\ttrain_loss: 1.3589\n",
      "Iteration: 120 of 277\ttrain_loss: 1.3396\n",
      "Iteration: 122 of 277\ttrain_loss: 1.3581\n",
      "Iteration: 124 of 277\ttrain_loss: 1.3165\n",
      "Iteration: 126 of 277\ttrain_loss: 1.2704\n",
      "Iteration: 128 of 277\ttrain_loss: 1.3110\n",
      "Iteration: 130 of 277\ttrain_loss: 1.4053\n",
      "Iteration: 132 of 277\ttrain_loss: 1.3553\n",
      "Iteration: 134 of 277\ttrain_loss: 1.2846\n",
      "Iteration: 136 of 277\ttrain_loss: 1.4148\n",
      "Iteration: 138 of 277\ttrain_loss: 1.3040\n",
      "Iteration: 140 of 277\ttrain_loss: 1.6804\n",
      "Iteration: 142 of 277\ttrain_loss: 1.4678\n",
      "Iteration: 144 of 277\ttrain_loss: 1.3532\n",
      "Iteration: 146 of 277\ttrain_loss: 1.3776\n",
      "Iteration: 148 of 277\ttrain_loss: 1.1918\n",
      "Iteration: 150 of 277\ttrain_loss: 1.3859\n",
      "Iteration: 152 of 277\ttrain_loss: 1.4280\n",
      "Iteration: 154 of 277\ttrain_loss: 1.3306\n",
      "Iteration: 156 of 277\ttrain_loss: 1.4086\n",
      "Iteration: 158 of 277\ttrain_loss: 1.3154\n",
      "Iteration: 160 of 277\ttrain_loss: 1.3104\n",
      "Iteration: 162 of 277\ttrain_loss: 2.2357\n",
      "Iteration: 164 of 277\ttrain_loss: 1.4422\n",
      "Iteration: 166 of 277\ttrain_loss: 1.4703\n",
      "Iteration: 168 of 277\ttrain_loss: 1.3152\n",
      "Iteration: 170 of 277\ttrain_loss: 1.4510\n",
      "Iteration: 172 of 277\ttrain_loss: 1.4062\n",
      "Iteration: 174 of 277\ttrain_loss: 1.4112\n",
      "Iteration: 176 of 277\ttrain_loss: 1.2703\n",
      "Iteration: 178 of 277\ttrain_loss: 1.6032\n",
      "Iteration: 180 of 277\ttrain_loss: 1.4600\n",
      "Iteration: 182 of 277\ttrain_loss: 1.3688\n",
      "Iteration: 184 of 277\ttrain_loss: 1.3861\n",
      "Iteration: 186 of 277\ttrain_loss: 1.1992\n",
      "Iteration: 188 of 277\ttrain_loss: 1.2936\n",
      "Iteration: 190 of 277\ttrain_loss: 1.3744\n",
      "Iteration: 192 of 277\ttrain_loss: 1.3166\n",
      "Iteration: 194 of 277\ttrain_loss: 1.4431\n",
      "Iteration: 196 of 277\ttrain_loss: 1.5866\n",
      "Iteration: 198 of 277\ttrain_loss: 1.3597\n",
      "Iteration: 200 of 277\ttrain_loss: 1.4956\n",
      "Iteration: 202 of 277\ttrain_loss: 1.4615\n",
      "Iteration: 204 of 277\ttrain_loss: 1.2409\n",
      "Iteration: 206 of 277\ttrain_loss: 1.3126\n",
      "Iteration: 208 of 277\ttrain_loss: 1.2928\n",
      "Iteration: 210 of 277\ttrain_loss: 1.3893\n",
      "Iteration: 212 of 277\ttrain_loss: 1.2748\n",
      "Iteration: 214 of 277\ttrain_loss: 1.3306\n",
      "Iteration: 216 of 277\ttrain_loss: 1.2199\n",
      "Iteration: 218 of 277\ttrain_loss: 1.5037\n",
      "Iteration: 220 of 277\ttrain_loss: 1.2084\n",
      "Iteration: 222 of 277\ttrain_loss: 1.3379\n",
      "Iteration: 224 of 277\ttrain_loss: 1.3566\n",
      "Iteration: 226 of 277\ttrain_loss: 1.3650\n",
      "Iteration: 228 of 277\ttrain_loss: 1.3362\n",
      "Iteration: 230 of 277\ttrain_loss: 1.2682\n",
      "Iteration: 232 of 277\ttrain_loss: 1.3747\n",
      "Iteration: 234 of 277\ttrain_loss: 1.3545\n",
      "Iteration: 236 of 277\ttrain_loss: 1.3109\n",
      "Iteration: 238 of 277\ttrain_loss: 1.3926\n",
      "Iteration: 240 of 277\ttrain_loss: 1.5681\n",
      "Iteration: 242 of 277\ttrain_loss: 1.4221\n",
      "Iteration: 244 of 277\ttrain_loss: 1.2506\n",
      "Iteration: 246 of 277\ttrain_loss: 1.3300\n",
      "Iteration: 248 of 277\ttrain_loss: 1.3870\n",
      "Iteration: 250 of 277\ttrain_loss: 1.2470\n",
      "Iteration: 252 of 277\ttrain_loss: 1.3219\n",
      "Iteration: 254 of 277\ttrain_loss: 1.4026\n",
      "Iteration: 256 of 277\ttrain_loss: 1.3751\n",
      "Iteration: 258 of 277\ttrain_loss: 1.5406\n",
      "Iteration: 260 of 277\ttrain_loss: 1.1896\n",
      "Iteration: 262 of 277\ttrain_loss: 1.2677\n",
      "Iteration: 264 of 277\ttrain_loss: 1.4304\n",
      "Iteration: 266 of 277\ttrain_loss: 1.4049\n",
      "Iteration: 268 of 277\ttrain_loss: 1.5475\n",
      "Iteration: 270 of 277\ttrain_loss: 1.4719\n",
      "Iteration: 272 of 277\ttrain_loss: 1.6437\n",
      "Iteration: 274 of 277\ttrain_loss: 1.3819\n",
      "Iteration: 276 of 277\ttrain_loss: 1.3349\n",
      "Iteration: 277 of 277\ttrain_loss: 1.4584\n",
      "Average Score for this Epoch: 1.3530923128128052\n",
      "Eval_loss: 2.5255484580993652\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 21 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.3745\n",
      "Iteration: 2 of 277\ttrain_loss: 1.3974\n",
      "Iteration: 4 of 277\ttrain_loss: 1.1812\n",
      "Iteration: 6 of 277\ttrain_loss: 1.3030\n",
      "Iteration: 8 of 277\ttrain_loss: 1.2748\n",
      "Iteration: 10 of 277\ttrain_loss: 1.3568\n",
      "Iteration: 12 of 277\ttrain_loss: 1.2949\n",
      "Iteration: 14 of 277\ttrain_loss: 1.3390\n",
      "Iteration: 16 of 277\ttrain_loss: 1.3738\n",
      "Iteration: 18 of 277\ttrain_loss: 1.2524\n",
      "Iteration: 20 of 277\ttrain_loss: 1.4499\n",
      "Iteration: 22 of 277\ttrain_loss: 1.2718\n",
      "Iteration: 24 of 277\ttrain_loss: 1.2524\n",
      "Iteration: 26 of 277\ttrain_loss: 1.2748\n",
      "Iteration: 28 of 277\ttrain_loss: 1.2951\n",
      "Iteration: 30 of 277\ttrain_loss: 1.3725\n",
      "Iteration: 32 of 277\ttrain_loss: 1.3726\n",
      "Iteration: 34 of 277\ttrain_loss: 1.4298\n",
      "Iteration: 36 of 277\ttrain_loss: 1.2501\n",
      "Iteration: 38 of 277\ttrain_loss: 1.3044\n",
      "Iteration: 40 of 277\ttrain_loss: 1.2639\n",
      "Iteration: 42 of 277\ttrain_loss: 1.5413\n",
      "Iteration: 44 of 277\ttrain_loss: 1.1949\n",
      "Iteration: 46 of 277\ttrain_loss: 1.5799\n",
      "Iteration: 48 of 277\ttrain_loss: 1.3870\n",
      "Iteration: 50 of 277\ttrain_loss: 1.3728\n",
      "Iteration: 52 of 277\ttrain_loss: 1.2976\n",
      "Iteration: 54 of 277\ttrain_loss: 1.3221\n",
      "Iteration: 56 of 277\ttrain_loss: 1.2379\n",
      "Iteration: 58 of 277\ttrain_loss: 1.5151\n",
      "Iteration: 60 of 277\ttrain_loss: 1.2709\n",
      "Iteration: 62 of 277\ttrain_loss: 1.3879\n",
      "Iteration: 64 of 277\ttrain_loss: 1.3826\n",
      "Iteration: 66 of 277\ttrain_loss: 1.3698\n",
      "Iteration: 68 of 277\ttrain_loss: 1.5819\n",
      "Iteration: 70 of 277\ttrain_loss: 1.4714\n",
      "Iteration: 72 of 277\ttrain_loss: 1.2916\n",
      "Iteration: 74 of 277\ttrain_loss: 1.3988\n",
      "Iteration: 76 of 277\ttrain_loss: 1.1836\n",
      "Iteration: 78 of 277\ttrain_loss: 1.3647\n",
      "Iteration: 80 of 277\ttrain_loss: 1.2461\n",
      "Iteration: 82 of 277\ttrain_loss: 1.2973\n",
      "Iteration: 84 of 277\ttrain_loss: 1.5308\n",
      "Iteration: 86 of 277\ttrain_loss: 1.3107\n",
      "Iteration: 88 of 277\ttrain_loss: 1.3614\n",
      "Iteration: 90 of 277\ttrain_loss: 1.3178\n",
      "Iteration: 92 of 277\ttrain_loss: 1.3262\n",
      "Iteration: 94 of 277\ttrain_loss: 1.1992\n",
      "Iteration: 96 of 277\ttrain_loss: 1.5180\n",
      "Iteration: 98 of 277\ttrain_loss: 1.3533\n",
      "Iteration: 100 of 277\ttrain_loss: 1.5236\n",
      "Iteration: 102 of 277\ttrain_loss: 1.2411\n",
      "Iteration: 104 of 277\ttrain_loss: 1.2502\n",
      "Iteration: 106 of 277\ttrain_loss: 1.3310\n",
      "Iteration: 108 of 277\ttrain_loss: 1.3900\n",
      "Iteration: 110 of 277\ttrain_loss: 1.4686\n",
      "Iteration: 112 of 277\ttrain_loss: 1.1539\n",
      "Iteration: 114 of 277\ttrain_loss: 1.2438\n",
      "Iteration: 116 of 277\ttrain_loss: 1.3959\n",
      "Iteration: 118 of 277\ttrain_loss: 1.4461\n",
      "Iteration: 120 of 277\ttrain_loss: 1.3275\n",
      "Iteration: 122 of 277\ttrain_loss: 1.4956\n",
      "Iteration: 124 of 277\ttrain_loss: 1.3641\n",
      "Iteration: 126 of 277\ttrain_loss: 1.4680\n",
      "Iteration: 128 of 277\ttrain_loss: 1.4037\n",
      "Iteration: 130 of 277\ttrain_loss: 1.3306\n",
      "Iteration: 132 of 277\ttrain_loss: 1.4371\n",
      "Iteration: 134 of 277\ttrain_loss: 1.2502\n",
      "Iteration: 136 of 277\ttrain_loss: 1.2687\n",
      "Iteration: 138 of 277\ttrain_loss: 1.3645\n",
      "Iteration: 140 of 277\ttrain_loss: 1.3864\n",
      "Iteration: 142 of 277\ttrain_loss: 1.3654\n",
      "Iteration: 144 of 277\ttrain_loss: 1.2223\n",
      "Iteration: 146 of 277\ttrain_loss: 1.3160\n",
      "Iteration: 148 of 277\ttrain_loss: 1.4942\n",
      "Iteration: 150 of 277\ttrain_loss: 1.3133\n",
      "Iteration: 152 of 277\ttrain_loss: 1.4276\n",
      "Iteration: 154 of 277\ttrain_loss: 1.3338\n",
      "Iteration: 156 of 277\ttrain_loss: 1.4026\n",
      "Iteration: 158 of 277\ttrain_loss: 1.4157\n",
      "Iteration: 160 of 277\ttrain_loss: 1.4229\n",
      "Iteration: 162 of 277\ttrain_loss: 1.2645\n",
      "Iteration: 164 of 277\ttrain_loss: 1.4835\n",
      "Iteration: 166 of 277\ttrain_loss: 1.2790\n",
      "Iteration: 168 of 277\ttrain_loss: 1.5290\n",
      "Iteration: 170 of 277\ttrain_loss: 1.4063\n",
      "Iteration: 172 of 277\ttrain_loss: 1.2208\n",
      "Iteration: 174 of 277\ttrain_loss: 1.3000\n",
      "Iteration: 176 of 277\ttrain_loss: 1.3620\n",
      "Iteration: 178 of 277\ttrain_loss: 1.3220\n",
      "Iteration: 180 of 277\ttrain_loss: 1.3689\n",
      "Iteration: 182 of 277\ttrain_loss: 1.4941\n",
      "Iteration: 184 of 277\ttrain_loss: 1.2996\n",
      "Iteration: 186 of 277\ttrain_loss: 1.4092\n",
      "Iteration: 188 of 277\ttrain_loss: 1.4066\n",
      "Iteration: 190 of 277\ttrain_loss: 1.3721\n",
      "Iteration: 192 of 277\ttrain_loss: 1.4090\n",
      "Iteration: 194 of 277\ttrain_loss: 1.2927\n",
      "Iteration: 196 of 277\ttrain_loss: 1.3444\n",
      "Iteration: 198 of 277\ttrain_loss: 1.5040\n",
      "Iteration: 200 of 277\ttrain_loss: 1.3491\n",
      "Iteration: 202 of 277\ttrain_loss: 1.5429\n",
      "Iteration: 204 of 277\ttrain_loss: 1.4793\n",
      "Iteration: 206 of 277\ttrain_loss: 1.3298\n",
      "Iteration: 208 of 277\ttrain_loss: 1.3594\n",
      "Iteration: 210 of 277\ttrain_loss: 1.3845\n",
      "Iteration: 212 of 277\ttrain_loss: 1.2416\n",
      "Iteration: 214 of 277\ttrain_loss: 1.4292\n",
      "Iteration: 216 of 277\ttrain_loss: 1.5557\n",
      "Iteration: 218 of 277\ttrain_loss: 1.5331\n",
      "Iteration: 220 of 277\ttrain_loss: 1.5442\n",
      "Iteration: 222 of 277\ttrain_loss: 1.3719\n",
      "Iteration: 224 of 277\ttrain_loss: 1.5441\n",
      "Iteration: 226 of 277\ttrain_loss: 1.3561\n",
      "Iteration: 228 of 277\ttrain_loss: 1.5122\n",
      "Iteration: 230 of 277\ttrain_loss: 1.3605\n",
      "Iteration: 232 of 277\ttrain_loss: 1.2913\n",
      "Iteration: 234 of 277\ttrain_loss: 1.4324\n",
      "Iteration: 236 of 277\ttrain_loss: 1.4631\n",
      "Iteration: 238 of 277\ttrain_loss: 1.4271\n",
      "Iteration: 240 of 277\ttrain_loss: 1.2781\n",
      "Iteration: 242 of 277\ttrain_loss: 1.4892\n",
      "Iteration: 244 of 277\ttrain_loss: 1.2481\n",
      "Iteration: 246 of 277\ttrain_loss: 1.2752\n",
      "Iteration: 248 of 277\ttrain_loss: 1.3764\n",
      "Iteration: 250 of 277\ttrain_loss: 1.3501\n",
      "Iteration: 252 of 277\ttrain_loss: 1.5415\n",
      "Iteration: 254 of 277\ttrain_loss: 1.5122\n",
      "Iteration: 256 of 277\ttrain_loss: 1.5028\n",
      "Iteration: 258 of 277\ttrain_loss: 1.3928\n",
      "Iteration: 260 of 277\ttrain_loss: 1.3611\n",
      "Iteration: 262 of 277\ttrain_loss: 1.4768\n",
      "Iteration: 264 of 277\ttrain_loss: 1.6975\n",
      "Iteration: 266 of 277\ttrain_loss: 1.5258\n",
      "Iteration: 268 of 277\ttrain_loss: 1.5232\n",
      "Iteration: 270 of 277\ttrain_loss: 1.4657\n",
      "Iteration: 272 of 277\ttrain_loss: 1.4916\n",
      "Iteration: 274 of 277\ttrain_loss: 1.3917\n",
      "Iteration: 276 of 277\ttrain_loss: 1.4454\n",
      "Iteration: 277 of 277\ttrain_loss: 1.4946\n",
      "Average Score for this Epoch: 1.382359504699707\n",
      "Eval_loss: 2.5657482147216797\n",
      "\n",
      "-------------------- Epoch 22 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.3738\n",
      "Iteration: 2 of 277\ttrain_loss: 1.4008\n",
      "Iteration: 4 of 277\ttrain_loss: 1.4922\n",
      "Iteration: 6 of 277\ttrain_loss: 1.5367\n",
      "Iteration: 8 of 277\ttrain_loss: 1.2934\n",
      "Iteration: 10 of 277\ttrain_loss: 1.3547\n",
      "Iteration: 12 of 277\ttrain_loss: 1.3560\n",
      "Iteration: 14 of 277\ttrain_loss: 1.3142\n",
      "Iteration: 16 of 277\ttrain_loss: 1.3592\n",
      "Iteration: 18 of 277\ttrain_loss: 1.4129\n",
      "Iteration: 20 of 277\ttrain_loss: 1.1779\n",
      "Iteration: 22 of 277\ttrain_loss: 1.2315\n",
      "Iteration: 24 of 277\ttrain_loss: 1.3022\n",
      "Iteration: 26 of 277\ttrain_loss: 1.4811\n",
      "Iteration: 28 of 277\ttrain_loss: 1.2689\n",
      "Iteration: 30 of 277\ttrain_loss: 1.2456\n",
      "Iteration: 32 of 277\ttrain_loss: 1.2708\n",
      "Iteration: 34 of 277\ttrain_loss: 1.4476\n",
      "Iteration: 36 of 277\ttrain_loss: 1.6232\n",
      "Iteration: 38 of 277\ttrain_loss: 1.3848\n",
      "Iteration: 40 of 277\ttrain_loss: 1.3741\n",
      "Iteration: 42 of 277\ttrain_loss: 1.2061\n",
      "Iteration: 44 of 277\ttrain_loss: 1.3063\n",
      "Iteration: 46 of 277\ttrain_loss: 1.4044\n",
      "Iteration: 48 of 277\ttrain_loss: 1.4263\n",
      "Iteration: 50 of 277\ttrain_loss: 1.2161\n",
      "Iteration: 52 of 277\ttrain_loss: 1.3734\n",
      "Iteration: 54 of 277\ttrain_loss: 1.4081\n",
      "Iteration: 56 of 277\ttrain_loss: 1.3894\n",
      "Iteration: 58 of 277\ttrain_loss: 1.2534\n",
      "Iteration: 60 of 277\ttrain_loss: 1.2857\n",
      "Iteration: 62 of 277\ttrain_loss: 1.3693\n",
      "Iteration: 64 of 277\ttrain_loss: 1.4295\n",
      "Iteration: 66 of 277\ttrain_loss: 1.2790\n",
      "Iteration: 68 of 277\ttrain_loss: 1.4219\n",
      "Iteration: 70 of 277\ttrain_loss: 1.2964\n",
      "Iteration: 72 of 277\ttrain_loss: 1.2764\n",
      "Iteration: 74 of 277\ttrain_loss: 1.6196\n",
      "Iteration: 76 of 277\ttrain_loss: 1.4917\n",
      "Iteration: 78 of 277\ttrain_loss: 1.4516\n",
      "Iteration: 80 of 277\ttrain_loss: 1.4139\n",
      "Iteration: 82 of 277\ttrain_loss: 1.4095\n",
      "Iteration: 84 of 277\ttrain_loss: 1.3985\n",
      "Iteration: 86 of 277\ttrain_loss: 1.3491\n",
      "Iteration: 88 of 277\ttrain_loss: 1.4638\n",
      "Iteration: 90 of 277\ttrain_loss: 1.5826\n",
      "Iteration: 92 of 277\ttrain_loss: 1.2655\n",
      "Iteration: 94 of 277\ttrain_loss: 1.4660\n",
      "Iteration: 96 of 277\ttrain_loss: 1.4533\n",
      "Iteration: 98 of 277\ttrain_loss: 1.2964\n",
      "Iteration: 100 of 277\ttrain_loss: 1.3663\n",
      "Iteration: 102 of 277\ttrain_loss: 1.5338\n",
      "Iteration: 104 of 277\ttrain_loss: 1.2610\n",
      "Iteration: 106 of 277\ttrain_loss: 1.2455\n",
      "Iteration: 108 of 277\ttrain_loss: 1.3988\n",
      "Iteration: 110 of 277\ttrain_loss: 1.4765\n",
      "Iteration: 112 of 277\ttrain_loss: 1.4545\n",
      "Iteration: 114 of 277\ttrain_loss: 1.4535\n",
      "Iteration: 116 of 277\ttrain_loss: 1.5510\n",
      "Iteration: 118 of 277\ttrain_loss: 1.2078\n",
      "Iteration: 120 of 277\ttrain_loss: 1.1986\n",
      "Iteration: 122 of 277\ttrain_loss: 1.5362\n",
      "Iteration: 124 of 277\ttrain_loss: 1.2867\n",
      "Iteration: 126 of 277\ttrain_loss: 1.3925\n",
      "Iteration: 128 of 277\ttrain_loss: 1.4766\n",
      "Iteration: 130 of 277\ttrain_loss: 1.3500\n",
      "Iteration: 132 of 277\ttrain_loss: 1.3248\n",
      "Iteration: 134 of 277\ttrain_loss: 1.5251\n",
      "Iteration: 136 of 277\ttrain_loss: 1.4826\n",
      "Iteration: 138 of 277\ttrain_loss: 1.3782\n",
      "Iteration: 140 of 277\ttrain_loss: 1.4152\n",
      "Iteration: 142 of 277\ttrain_loss: 1.4227\n",
      "Iteration: 144 of 277\ttrain_loss: 1.4842\n",
      "Iteration: 146 of 277\ttrain_loss: 1.3985\n",
      "Iteration: 148 of 277\ttrain_loss: 1.4990\n",
      "Iteration: 150 of 277\ttrain_loss: 1.3852\n",
      "Iteration: 152 of 277\ttrain_loss: 1.5205\n",
      "Iteration: 154 of 277\ttrain_loss: 1.3233\n",
      "Iteration: 156 of 277\ttrain_loss: 1.4035\n",
      "Iteration: 158 of 277\ttrain_loss: 1.5661\n",
      "Iteration: 160 of 277\ttrain_loss: 1.5008\n",
      "Iteration: 162 of 277\ttrain_loss: 1.3819\n",
      "Iteration: 164 of 277\ttrain_loss: 1.4503\n",
      "Iteration: 166 of 277\ttrain_loss: 1.3877\n",
      "Iteration: 168 of 277\ttrain_loss: 1.5640\n",
      "Iteration: 170 of 277\ttrain_loss: 1.5163\n",
      "Iteration: 172 of 277\ttrain_loss: 1.6637\n",
      "Iteration: 174 of 277\ttrain_loss: 1.4553\n",
      "Iteration: 176 of 277\ttrain_loss: 1.2526\n",
      "Iteration: 178 of 277\ttrain_loss: 1.4649\n",
      "Iteration: 180 of 277\ttrain_loss: 1.3711\n",
      "Iteration: 182 of 277\ttrain_loss: 1.3134\n",
      "Iteration: 184 of 277\ttrain_loss: 1.3677\n",
      "Iteration: 186 of 277\ttrain_loss: 1.4259\n",
      "Iteration: 188 of 277\ttrain_loss: 1.4755\n",
      "Iteration: 190 of 277\ttrain_loss: 1.3695\n",
      "Iteration: 192 of 277\ttrain_loss: 1.5585\n",
      "Iteration: 194 of 277\ttrain_loss: 1.6013\n",
      "Iteration: 196 of 277\ttrain_loss: 1.3210\n",
      "Iteration: 198 of 277\ttrain_loss: 1.5523\n",
      "Iteration: 200 of 277\ttrain_loss: 1.6606\n",
      "Iteration: 202 of 277\ttrain_loss: 1.5806\n",
      "Iteration: 204 of 277\ttrain_loss: 1.5668\n",
      "Iteration: 206 of 277\ttrain_loss: 1.3197\n",
      "Iteration: 208 of 277\ttrain_loss: 1.4788\n",
      "Iteration: 210 of 277\ttrain_loss: 1.5671\n",
      "Iteration: 212 of 277\ttrain_loss: 1.5562\n",
      "Iteration: 214 of 277\ttrain_loss: 1.3787\n",
      "Iteration: 216 of 277\ttrain_loss: 1.4463\n",
      "Iteration: 218 of 277\ttrain_loss: 1.5190\n",
      "Iteration: 220 of 277\ttrain_loss: 1.3433\n",
      "Iteration: 222 of 277\ttrain_loss: 1.4471\n",
      "Iteration: 224 of 277\ttrain_loss: 1.5855\n",
      "Iteration: 226 of 277\ttrain_loss: 1.6604\n",
      "Iteration: 228 of 277\ttrain_loss: 1.4812\n",
      "Iteration: 230 of 277\ttrain_loss: 1.6207\n",
      "Iteration: 232 of 277\ttrain_loss: 1.5523\n",
      "Iteration: 234 of 277\ttrain_loss: 1.4392\n",
      "Iteration: 236 of 277\ttrain_loss: 1.6541\n",
      "Iteration: 238 of 277\ttrain_loss: 1.5120\n",
      "Iteration: 240 of 277\ttrain_loss: 1.4964\n",
      "Iteration: 242 of 277\ttrain_loss: 1.5764\n",
      "Iteration: 244 of 277\ttrain_loss: 1.5237\n",
      "Iteration: 246 of 277\ttrain_loss: 1.6084\n",
      "Iteration: 248 of 277\ttrain_loss: 1.4845\n",
      "Iteration: 250 of 277\ttrain_loss: 1.4218\n",
      "Iteration: 252 of 277\ttrain_loss: 1.4627\n",
      "Iteration: 254 of 277\ttrain_loss: 1.4631\n",
      "Iteration: 256 of 277\ttrain_loss: 1.3169\n",
      "Iteration: 258 of 277\ttrain_loss: 1.5521\n",
      "Iteration: 260 of 277\ttrain_loss: 1.4111\n",
      "Iteration: 262 of 277\ttrain_loss: 1.4117\n",
      "Iteration: 264 of 277\ttrain_loss: 1.4852\n",
      "Iteration: 266 of 277\ttrain_loss: 1.5848\n",
      "Iteration: 268 of 277\ttrain_loss: 1.5902\n",
      "Iteration: 270 of 277\ttrain_loss: 1.4260\n",
      "Iteration: 272 of 277\ttrain_loss: 1.4130\n",
      "Iteration: 274 of 277\ttrain_loss: 1.5608\n",
      "Iteration: 276 of 277\ttrain_loss: 1.2628\n",
      "Iteration: 277 of 277\ttrain_loss: 1.3171\n",
      "Average Score for this Epoch: 1.4348326921463013\n",
      "Eval_loss: 2.5795035362243652\n",
      "\n",
      "-------------------- Epoch 23 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.2720\n",
      "Iteration: 2 of 277\ttrain_loss: 1.3144\n",
      "Iteration: 4 of 277\ttrain_loss: 1.4001\n",
      "Iteration: 6 of 277\ttrain_loss: 1.3132\n",
      "Iteration: 8 of 277\ttrain_loss: 1.2558\n",
      "Iteration: 10 of 277\ttrain_loss: 1.2505\n",
      "Iteration: 12 of 277\ttrain_loss: 1.3332\n",
      "Iteration: 14 of 277\ttrain_loss: 1.2983\n",
      "Iteration: 16 of 277\ttrain_loss: 1.3106\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1955\n",
      "Iteration: 20 of 277\ttrain_loss: 1.2429\n",
      "Iteration: 22 of 277\ttrain_loss: 1.3601\n",
      "Iteration: 24 of 277\ttrain_loss: 1.3949\n",
      "Iteration: 26 of 277\ttrain_loss: 1.4270\n",
      "Iteration: 28 of 277\ttrain_loss: 1.2501\n",
      "Iteration: 30 of 277\ttrain_loss: 1.3093\n",
      "Iteration: 32 of 277\ttrain_loss: 1.4351\n",
      "Iteration: 34 of 277\ttrain_loss: 1.3268\n",
      "Iteration: 36 of 277\ttrain_loss: 1.3073\n",
      "Iteration: 38 of 277\ttrain_loss: 1.3346\n",
      "Iteration: 40 of 277\ttrain_loss: 1.4499\n",
      "Iteration: 42 of 277\ttrain_loss: 1.2950\n",
      "Iteration: 44 of 277\ttrain_loss: 1.3545\n",
      "Iteration: 46 of 277\ttrain_loss: 1.6801\n",
      "Iteration: 48 of 277\ttrain_loss: 1.3348\n",
      "Iteration: 50 of 277\ttrain_loss: 1.2708\n",
      "Iteration: 52 of 277\ttrain_loss: 1.5216\n",
      "Iteration: 54 of 277\ttrain_loss: 1.3597\n",
      "Iteration: 56 of 277\ttrain_loss: 1.4415\n",
      "Iteration: 58 of 277\ttrain_loss: 1.3234\n",
      "Iteration: 60 of 277\ttrain_loss: 1.2877\n",
      "Iteration: 62 of 277\ttrain_loss: 1.3469\n",
      "Iteration: 64 of 277\ttrain_loss: 1.2696\n",
      "Iteration: 66 of 277\ttrain_loss: 1.4714\n",
      "Iteration: 68 of 277\ttrain_loss: 1.3031\n",
      "Iteration: 70 of 277\ttrain_loss: 1.4453\n",
      "Iteration: 72 of 277\ttrain_loss: 1.2899\n",
      "Iteration: 74 of 277\ttrain_loss: 1.3312\n",
      "Iteration: 76 of 277\ttrain_loss: 1.6180\n",
      "Iteration: 78 of 277\ttrain_loss: 1.5362\n",
      "Iteration: 80 of 277\ttrain_loss: 1.5205\n",
      "Iteration: 82 of 277\ttrain_loss: 1.3363\n",
      "Iteration: 84 of 277\ttrain_loss: 1.5365\n",
      "Iteration: 86 of 277\ttrain_loss: 1.4002\n",
      "Iteration: 88 of 277\ttrain_loss: 1.3572\n",
      "Iteration: 90 of 277\ttrain_loss: 1.3872\n",
      "Iteration: 92 of 277\ttrain_loss: 1.4381\n",
      "Iteration: 94 of 277\ttrain_loss: 1.3651\n",
      "Iteration: 96 of 277\ttrain_loss: 1.4112\n",
      "Iteration: 98 of 277\ttrain_loss: 1.3417\n",
      "Iteration: 100 of 277\ttrain_loss: 1.3802\n",
      "Iteration: 102 of 277\ttrain_loss: 1.2557\n",
      "Iteration: 104 of 277\ttrain_loss: 1.4630\n",
      "Iteration: 106 of 277\ttrain_loss: 1.3989\n",
      "Iteration: 108 of 277\ttrain_loss: 1.3168\n",
      "Iteration: 110 of 277\ttrain_loss: 1.4095\n",
      "Iteration: 112 of 277\ttrain_loss: 1.2637\n",
      "Iteration: 114 of 277\ttrain_loss: 1.3908\n",
      "Iteration: 116 of 277\ttrain_loss: 1.3207\n",
      "Iteration: 118 of 277\ttrain_loss: 1.3036\n",
      "Iteration: 120 of 277\ttrain_loss: 1.4008\n",
      "Iteration: 122 of 277\ttrain_loss: 1.2723\n",
      "Iteration: 124 of 277\ttrain_loss: 1.3961\n",
      "Iteration: 126 of 277\ttrain_loss: 1.3066\n",
      "Iteration: 128 of 277\ttrain_loss: 1.4510\n",
      "Iteration: 130 of 277\ttrain_loss: 1.5636\n",
      "Iteration: 132 of 277\ttrain_loss: 1.4757\n",
      "Iteration: 134 of 277\ttrain_loss: 1.4933\n",
      "Iteration: 136 of 277\ttrain_loss: 1.1493\n",
      "Iteration: 138 of 277\ttrain_loss: 1.3535\n",
      "Iteration: 140 of 277\ttrain_loss: 1.1452\n",
      "Iteration: 142 of 277\ttrain_loss: 1.2563\n",
      "Iteration: 144 of 277\ttrain_loss: 1.3752\n",
      "Iteration: 146 of 277\ttrain_loss: 1.3848\n",
      "Iteration: 148 of 277\ttrain_loss: 1.4223\n",
      "Iteration: 150 of 277\ttrain_loss: 1.3301\n",
      "Iteration: 152 of 277\ttrain_loss: 1.3649\n",
      "Iteration: 154 of 277\ttrain_loss: 1.6505\n",
      "Iteration: 156 of 277\ttrain_loss: 1.3749\n",
      "Iteration: 158 of 277\ttrain_loss: 1.2824\n",
      "Iteration: 160 of 277\ttrain_loss: 1.2832\n",
      "Iteration: 162 of 277\ttrain_loss: 1.5651\n",
      "Iteration: 164 of 277\ttrain_loss: 1.4653\n",
      "Iteration: 166 of 277\ttrain_loss: 1.3455\n",
      "Iteration: 168 of 277\ttrain_loss: 1.2941\n",
      "Iteration: 170 of 277\ttrain_loss: 1.3064\n",
      "Iteration: 172 of 277\ttrain_loss: 1.4045\n",
      "Iteration: 174 of 277\ttrain_loss: 1.3523\n",
      "Iteration: 176 of 277\ttrain_loss: 1.2699\n",
      "Iteration: 178 of 277\ttrain_loss: 1.4398\n",
      "Iteration: 180 of 277\ttrain_loss: 1.4271\n",
      "Iteration: 182 of 277\ttrain_loss: 1.3245\n",
      "Iteration: 184 of 277\ttrain_loss: 1.4100\n",
      "Iteration: 186 of 277\ttrain_loss: 1.2880\n",
      "Iteration: 188 of 277\ttrain_loss: 1.3841\n",
      "Iteration: 190 of 277\ttrain_loss: 1.1967\n",
      "Iteration: 192 of 277\ttrain_loss: 1.2335\n",
      "Iteration: 194 of 277\ttrain_loss: 1.2454\n",
      "Iteration: 196 of 277\ttrain_loss: 1.4460\n",
      "Iteration: 198 of 277\ttrain_loss: 1.4510\n",
      "Iteration: 200 of 277\ttrain_loss: 1.3396\n",
      "Iteration: 202 of 277\ttrain_loss: 1.2968\n",
      "Iteration: 204 of 277\ttrain_loss: 1.4854\n",
      "Iteration: 206 of 277\ttrain_loss: 1.3271\n",
      "Iteration: 208 of 277\ttrain_loss: 1.5041\n",
      "Iteration: 210 of 277\ttrain_loss: 1.3870\n",
      "Iteration: 212 of 277\ttrain_loss: 1.3647\n",
      "Iteration: 214 of 277\ttrain_loss: 1.2404\n",
      "Iteration: 216 of 277\ttrain_loss: 1.2657\n",
      "Iteration: 218 of 277\ttrain_loss: 1.4395\n",
      "Iteration: 220 of 277\ttrain_loss: 1.3405\n",
      "Iteration: 222 of 277\ttrain_loss: 1.3396\n",
      "Iteration: 224 of 277\ttrain_loss: 1.4184\n",
      "Iteration: 226 of 277\ttrain_loss: 1.2631\n",
      "Iteration: 228 of 277\ttrain_loss: 1.4359\n",
      "Iteration: 230 of 277\ttrain_loss: 1.3044\n",
      "Iteration: 232 of 277\ttrain_loss: 1.3465\n",
      "Iteration: 234 of 277\ttrain_loss: 1.3898\n",
      "Iteration: 236 of 277\ttrain_loss: 1.3942\n",
      "Iteration: 238 of 277\ttrain_loss: 1.2506\n",
      "Iteration: 240 of 277\ttrain_loss: 1.2611\n",
      "Iteration: 242 of 277\ttrain_loss: 1.5761\n",
      "Iteration: 244 of 277\ttrain_loss: 1.3145\n",
      "Iteration: 246 of 277\ttrain_loss: 1.4799\n",
      "Iteration: 248 of 277\ttrain_loss: 1.4732\n",
      "Iteration: 250 of 277\ttrain_loss: 1.4141\n",
      "Iteration: 252 of 277\ttrain_loss: 1.3594\n",
      "Iteration: 254 of 277\ttrain_loss: 1.3407\n",
      "Iteration: 256 of 277\ttrain_loss: 1.3671\n",
      "Iteration: 258 of 277\ttrain_loss: 1.4493\n",
      "Iteration: 260 of 277\ttrain_loss: 1.2633\n",
      "Iteration: 262 of 277\ttrain_loss: 1.4186\n",
      "Iteration: 264 of 277\ttrain_loss: 1.5251\n",
      "Iteration: 266 of 277\ttrain_loss: 1.3238\n",
      "Iteration: 268 of 277\ttrain_loss: 1.4098\n",
      "Iteration: 270 of 277\ttrain_loss: 1.4198\n",
      "Iteration: 272 of 277\ttrain_loss: 1.2824\n",
      "Iteration: 274 of 277\ttrain_loss: 1.4551\n",
      "Iteration: 276 of 277\ttrain_loss: 1.4098\n",
      "Iteration: 277 of 277\ttrain_loss: 1.2698\n",
      "Average Score for this Epoch: 1.3792489767074585\n",
      "Eval_loss: 2.527113676071167\n",
      "\n",
      "-------------------- Epoch 24 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.2617\n",
      "Iteration: 2 of 277\ttrain_loss: 1.2076\n",
      "Iteration: 4 of 277\ttrain_loss: 1.1253\n",
      "Iteration: 6 of 277\ttrain_loss: 1.2969\n",
      "Iteration: 8 of 277\ttrain_loss: 1.3777\n",
      "Iteration: 10 of 277\ttrain_loss: 1.1810\n",
      "Iteration: 12 of 277\ttrain_loss: 1.1809\n",
      "Iteration: 14 of 277\ttrain_loss: 1.2266\n",
      "Iteration: 16 of 277\ttrain_loss: 1.1208\n",
      "Iteration: 18 of 277\ttrain_loss: 1.3041\n",
      "Iteration: 20 of 277\ttrain_loss: 1.0892\n",
      "Iteration: 22 of 277\ttrain_loss: 1.2735\n",
      "Iteration: 24 of 277\ttrain_loss: 1.2795\n",
      "Iteration: 26 of 277\ttrain_loss: 1.2316\n",
      "Iteration: 28 of 277\ttrain_loss: 1.3617\n",
      "Iteration: 30 of 277\ttrain_loss: 1.3159\n",
      "Iteration: 32 of 277\ttrain_loss: 1.2740\n",
      "Iteration: 34 of 277\ttrain_loss: 1.3572\n",
      "Iteration: 36 of 277\ttrain_loss: 1.2240\n",
      "Iteration: 38 of 277\ttrain_loss: 1.2975\n",
      "Iteration: 40 of 277\ttrain_loss: 1.2868\n",
      "Iteration: 42 of 277\ttrain_loss: 1.2913\n",
      "Iteration: 44 of 277\ttrain_loss: 1.2905\n",
      "Iteration: 46 of 277\ttrain_loss: 1.2636\n",
      "Iteration: 48 of 277\ttrain_loss: 1.3937\n",
      "Iteration: 50 of 277\ttrain_loss: 1.2571\n",
      "Iteration: 52 of 277\ttrain_loss: 1.2287\n",
      "Iteration: 54 of 277\ttrain_loss: 1.3148\n",
      "Iteration: 56 of 277\ttrain_loss: 1.3271\n",
      "Iteration: 58 of 277\ttrain_loss: 1.1158\n",
      "Iteration: 60 of 277\ttrain_loss: 1.3160\n",
      "Iteration: 62 of 277\ttrain_loss: 1.2295\n",
      "Iteration: 64 of 277\ttrain_loss: 1.1631\n",
      "Iteration: 66 of 277\ttrain_loss: 1.3144\n",
      "Iteration: 68 of 277\ttrain_loss: 1.2854\n",
      "Iteration: 70 of 277\ttrain_loss: 1.1784\n",
      "Iteration: 72 of 277\ttrain_loss: 1.3076\n",
      "Iteration: 74 of 277\ttrain_loss: 1.3175\n",
      "Iteration: 76 of 277\ttrain_loss: 1.3816\n",
      "Iteration: 78 of 277\ttrain_loss: 1.3203\n",
      "Iteration: 80 of 277\ttrain_loss: 1.1371\n",
      "Iteration: 82 of 277\ttrain_loss: 1.2588\n",
      "Iteration: 84 of 277\ttrain_loss: 1.1214\n",
      "Iteration: 86 of 277\ttrain_loss: 1.3481\n",
      "Iteration: 88 of 277\ttrain_loss: 1.2614\n",
      "Iteration: 90 of 277\ttrain_loss: 1.0864\n",
      "Iteration: 92 of 277\ttrain_loss: 1.1585\n",
      "Iteration: 94 of 277\ttrain_loss: 1.2247\n",
      "Iteration: 96 of 277\ttrain_loss: 1.2436\n",
      "Iteration: 98 of 277\ttrain_loss: 1.1091\n",
      "Iteration: 100 of 277\ttrain_loss: 1.0862\n",
      "Iteration: 102 of 277\ttrain_loss: 1.2733\n",
      "Iteration: 104 of 277\ttrain_loss: 1.1537\n",
      "Iteration: 106 of 277\ttrain_loss: 1.4464\n",
      "Iteration: 108 of 277\ttrain_loss: 1.2123\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0919\n",
      "Iteration: 112 of 277\ttrain_loss: 1.3659\n",
      "Iteration: 114 of 277\ttrain_loss: 1.2124\n",
      "Iteration: 116 of 277\ttrain_loss: 1.3457\n",
      "Iteration: 118 of 277\ttrain_loss: 1.2213\n",
      "Iteration: 120 of 277\ttrain_loss: 1.3300\n",
      "Iteration: 122 of 277\ttrain_loss: 1.1653\n",
      "Iteration: 124 of 277\ttrain_loss: 1.3456\n",
      "Iteration: 126 of 277\ttrain_loss: 1.5406\n",
      "Iteration: 128 of 277\ttrain_loss: 1.2504\n",
      "Iteration: 130 of 277\ttrain_loss: 1.2012\n",
      "Iteration: 132 of 277\ttrain_loss: 1.0633\n",
      "Iteration: 134 of 277\ttrain_loss: 1.1932\n",
      "Iteration: 136 of 277\ttrain_loss: 1.4413\n",
      "Iteration: 138 of 277\ttrain_loss: 1.2279\n",
      "Iteration: 140 of 277\ttrain_loss: 1.3845\n",
      "Iteration: 142 of 277\ttrain_loss: 1.4780\n",
      "Iteration: 144 of 277\ttrain_loss: 1.2286\n",
      "Iteration: 146 of 277\ttrain_loss: 1.2698\n",
      "Iteration: 148 of 277\ttrain_loss: 1.2284\n",
      "Iteration: 150 of 277\ttrain_loss: 1.2145\n",
      "Iteration: 152 of 277\ttrain_loss: 1.2498\n",
      "Iteration: 154 of 277\ttrain_loss: 1.2855\n",
      "Iteration: 156 of 277\ttrain_loss: 1.3098\n",
      "Iteration: 158 of 277\ttrain_loss: 1.5372\n",
      "Iteration: 160 of 277\ttrain_loss: 1.2231\n",
      "Iteration: 162 of 277\ttrain_loss: 1.3055\n",
      "Iteration: 164 of 277\ttrain_loss: 1.4911\n",
      "Iteration: 166 of 277\ttrain_loss: 1.1910\n",
      "Iteration: 168 of 277\ttrain_loss: 1.2163\n",
      "Iteration: 170 of 277\ttrain_loss: 1.2480\n",
      "Iteration: 172 of 277\ttrain_loss: 1.2788\n",
      "Iteration: 174 of 277\ttrain_loss: 1.2143\n",
      "Iteration: 176 of 277\ttrain_loss: 1.2400\n",
      "Iteration: 178 of 277\ttrain_loss: 1.1047\n",
      "Iteration: 180 of 277\ttrain_loss: 1.2373\n",
      "Iteration: 182 of 277\ttrain_loss: 1.3118\n",
      "Iteration: 184 of 277\ttrain_loss: 1.3388\n",
      "Iteration: 186 of 277\ttrain_loss: 1.3001\n",
      "Iteration: 188 of 277\ttrain_loss: 1.3007\n",
      "Iteration: 190 of 277\ttrain_loss: 1.3136\n",
      "Iteration: 192 of 277\ttrain_loss: 1.5694\n",
      "Iteration: 194 of 277\ttrain_loss: 1.2419\n",
      "Iteration: 196 of 277\ttrain_loss: 1.2312\n",
      "Iteration: 198 of 277\ttrain_loss: 1.4478\n",
      "Iteration: 200 of 277\ttrain_loss: 1.3495\n",
      "Iteration: 202 of 277\ttrain_loss: 1.3098\n",
      "Iteration: 204 of 277\ttrain_loss: 1.3147\n",
      "Iteration: 206 of 277\ttrain_loss: 1.1428\n",
      "Iteration: 208 of 277\ttrain_loss: 1.4053\n",
      "Iteration: 210 of 277\ttrain_loss: 1.0958\n",
      "Iteration: 212 of 277\ttrain_loss: 1.3065\n",
      "Iteration: 214 of 277\ttrain_loss: 1.1466\n",
      "Iteration: 216 of 277\ttrain_loss: 1.2952\n",
      "Iteration: 218 of 277\ttrain_loss: 1.2868\n",
      "Iteration: 220 of 277\ttrain_loss: 1.2964\n",
      "Iteration: 222 of 277\ttrain_loss: 1.3843\n",
      "Iteration: 224 of 277\ttrain_loss: 1.1926\n",
      "Iteration: 226 of 277\ttrain_loss: 1.3236\n",
      "Iteration: 228 of 277\ttrain_loss: 1.3586\n",
      "Iteration: 230 of 277\ttrain_loss: 1.1558\n",
      "Iteration: 232 of 277\ttrain_loss: 1.3331\n",
      "Iteration: 234 of 277\ttrain_loss: 1.2559\n",
      "Iteration: 236 of 277\ttrain_loss: 1.1460\n",
      "Iteration: 238 of 277\ttrain_loss: 1.2126\n",
      "Iteration: 240 of 277\ttrain_loss: 1.3293\n",
      "Iteration: 242 of 277\ttrain_loss: 1.2596\n",
      "Iteration: 244 of 277\ttrain_loss: 1.5107\n",
      "Iteration: 246 of 277\ttrain_loss: 1.2716\n",
      "Iteration: 248 of 277\ttrain_loss: 1.1854\n",
      "Iteration: 250 of 277\ttrain_loss: 1.1211\n",
      "Iteration: 252 of 277\ttrain_loss: 1.3015\n",
      "Iteration: 254 of 277\ttrain_loss: 1.3542\n",
      "Iteration: 256 of 277\ttrain_loss: 1.3116\n",
      "Iteration: 258 of 277\ttrain_loss: 1.2402\n",
      "Iteration: 260 of 277\ttrain_loss: 1.1872\n",
      "Iteration: 262 of 277\ttrain_loss: 1.3039\n",
      "Iteration: 264 of 277\ttrain_loss: 1.1638\n",
      "Iteration: 266 of 277\ttrain_loss: 1.3363\n",
      "Iteration: 268 of 277\ttrain_loss: 1.0700\n",
      "Iteration: 270 of 277\ttrain_loss: 1.2812\n",
      "Iteration: 272 of 277\ttrain_loss: 1.0913\n",
      "Iteration: 274 of 277\ttrain_loss: 1.1123\n",
      "Iteration: 276 of 277\ttrain_loss: 1.1892\n",
      "Iteration: 277 of 277\ttrain_loss: 0.9962\n",
      "Average Score for this Epoch: 1.274743676185608\n",
      "Eval_loss: 2.513038396835327\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 25 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 0.9881\n",
      "Iteration: 2 of 277\ttrain_loss: 1.1044\n",
      "Iteration: 4 of 277\ttrain_loss: 1.2917\n",
      "Iteration: 6 of 277\ttrain_loss: 1.2358\n",
      "Iteration: 8 of 277\ttrain_loss: 1.0028\n",
      "Iteration: 10 of 277\ttrain_loss: 1.2008\n",
      "Iteration: 12 of 277\ttrain_loss: 1.1941\n",
      "Iteration: 14 of 277\ttrain_loss: 1.1447\n",
      "Iteration: 16 of 277\ttrain_loss: 1.1114\n",
      "Iteration: 18 of 277\ttrain_loss: 1.0990\n",
      "Iteration: 20 of 277\ttrain_loss: 1.3024\n",
      "Iteration: 22 of 277\ttrain_loss: 1.1389\n",
      "Iteration: 24 of 277\ttrain_loss: 1.1315\n",
      "Iteration: 26 of 277\ttrain_loss: 1.1969\n",
      "Iteration: 28 of 277\ttrain_loss: 1.1875\n",
      "Iteration: 30 of 277\ttrain_loss: 1.1854\n",
      "Iteration: 32 of 277\ttrain_loss: 1.0841\n",
      "Iteration: 34 of 277\ttrain_loss: 1.2134\n",
      "Iteration: 36 of 277\ttrain_loss: 1.1629\n",
      "Iteration: 38 of 277\ttrain_loss: 1.1628\n",
      "Iteration: 40 of 277\ttrain_loss: 1.1007\n",
      "Iteration: 42 of 277\ttrain_loss: 1.0944\n",
      "Iteration: 44 of 277\ttrain_loss: 1.2798\n",
      "Iteration: 46 of 277\ttrain_loss: 1.2580\n",
      "Iteration: 48 of 277\ttrain_loss: 1.1574\n",
      "Iteration: 50 of 277\ttrain_loss: 1.2253\n",
      "Iteration: 52 of 277\ttrain_loss: 1.1922\n",
      "Iteration: 54 of 277\ttrain_loss: 1.2780\n",
      "Iteration: 56 of 277\ttrain_loss: 1.1186\n",
      "Iteration: 58 of 277\ttrain_loss: 1.2667\n",
      "Iteration: 60 of 277\ttrain_loss: 1.2183\n",
      "Iteration: 62 of 277\ttrain_loss: 1.1814\n",
      "Iteration: 64 of 277\ttrain_loss: 1.1205\n",
      "Iteration: 66 of 277\ttrain_loss: 1.0980\n",
      "Iteration: 68 of 277\ttrain_loss: 1.1628\n",
      "Iteration: 70 of 277\ttrain_loss: 1.1857\n",
      "Iteration: 72 of 277\ttrain_loss: 1.4342\n",
      "Iteration: 74 of 277\ttrain_loss: 1.0779\n",
      "Iteration: 76 of 277\ttrain_loss: 1.0975\n",
      "Iteration: 78 of 277\ttrain_loss: 1.1867\n",
      "Iteration: 80 of 277\ttrain_loss: 1.3045\n",
      "Iteration: 82 of 277\ttrain_loss: 1.1936\n",
      "Iteration: 84 of 277\ttrain_loss: 1.1761\n",
      "Iteration: 86 of 277\ttrain_loss: 1.0256\n",
      "Iteration: 88 of 277\ttrain_loss: 1.1570\n",
      "Iteration: 90 of 277\ttrain_loss: 1.0777\n",
      "Iteration: 92 of 277\ttrain_loss: 1.2050\n",
      "Iteration: 94 of 277\ttrain_loss: 1.3533\n",
      "Iteration: 96 of 277\ttrain_loss: 1.1368\n",
      "Iteration: 98 of 277\ttrain_loss: 1.1196\n",
      "Iteration: 100 of 277\ttrain_loss: 1.3411\n",
      "Iteration: 102 of 277\ttrain_loss: 1.0450\n",
      "Iteration: 104 of 277\ttrain_loss: 1.1967\n",
      "Iteration: 106 of 277\ttrain_loss: 1.2360\n",
      "Iteration: 108 of 277\ttrain_loss: 1.1990\n",
      "Iteration: 110 of 277\ttrain_loss: 1.1018\n",
      "Iteration: 112 of 277\ttrain_loss: 1.1563\n",
      "Iteration: 114 of 277\ttrain_loss: 1.0240\n",
      "Iteration: 116 of 277\ttrain_loss: 1.2280\n",
      "Iteration: 118 of 277\ttrain_loss: 1.1545\n",
      "Iteration: 120 of 277\ttrain_loss: 1.1290\n",
      "Iteration: 122 of 277\ttrain_loss: 1.3270\n",
      "Iteration: 124 of 277\ttrain_loss: 1.1083\n",
      "Iteration: 126 of 277\ttrain_loss: 1.3083\n",
      "Iteration: 128 of 277\ttrain_loss: 1.1615\n",
      "Iteration: 130 of 277\ttrain_loss: 1.0637\n",
      "Iteration: 132 of 277\ttrain_loss: 1.2464\n",
      "Iteration: 134 of 277\ttrain_loss: 1.2883\n",
      "Iteration: 136 of 277\ttrain_loss: 1.1307\n",
      "Iteration: 138 of 277\ttrain_loss: 1.0307\n",
      "Iteration: 140 of 277\ttrain_loss: 1.1772\n",
      "Iteration: 142 of 277\ttrain_loss: 1.2023\n",
      "Iteration: 144 of 277\ttrain_loss: 1.2210\n",
      "Iteration: 146 of 277\ttrain_loss: 1.2924\n",
      "Iteration: 148 of 277\ttrain_loss: 1.2637\n",
      "Iteration: 150 of 277\ttrain_loss: 1.3966\n",
      "Iteration: 152 of 277\ttrain_loss: 1.1299\n",
      "Iteration: 154 of 277\ttrain_loss: 1.1683\n",
      "Iteration: 156 of 277\ttrain_loss: 1.5824\n",
      "Iteration: 158 of 277\ttrain_loss: 1.1729\n",
      "Iteration: 160 of 277\ttrain_loss: 1.1912\n",
      "Iteration: 162 of 277\ttrain_loss: 1.3795\n",
      "Iteration: 164 of 277\ttrain_loss: 1.2459\n",
      "Iteration: 166 of 277\ttrain_loss: 1.1865\n",
      "Iteration: 168 of 277\ttrain_loss: 1.1684\n",
      "Iteration: 170 of 277\ttrain_loss: 1.0316\n",
      "Iteration: 172 of 277\ttrain_loss: 1.2055\n",
      "Iteration: 174 of 277\ttrain_loss: 1.1820\n",
      "Iteration: 176 of 277\ttrain_loss: 1.2073\n",
      "Iteration: 178 of 277\ttrain_loss: 1.1577\n",
      "Iteration: 180 of 277\ttrain_loss: 1.2696\n",
      "Iteration: 182 of 277\ttrain_loss: 1.1881\n",
      "Iteration: 184 of 277\ttrain_loss: 1.2482\n",
      "Iteration: 186 of 277\ttrain_loss: 1.2008\n",
      "Iteration: 188 of 277\ttrain_loss: 1.0387\n",
      "Iteration: 190 of 277\ttrain_loss: 1.0907\n",
      "Iteration: 192 of 277\ttrain_loss: 1.1077\n",
      "Iteration: 194 of 277\ttrain_loss: 1.2057\n",
      "Iteration: 196 of 277\ttrain_loss: 1.3208\n",
      "Iteration: 198 of 277\ttrain_loss: 1.2664\n",
      "Iteration: 200 of 277\ttrain_loss: 1.3009\n",
      "Iteration: 202 of 277\ttrain_loss: 1.0446\n",
      "Iteration: 204 of 277\ttrain_loss: 1.0782\n",
      "Iteration: 206 of 277\ttrain_loss: 1.1971\n",
      "Iteration: 208 of 277\ttrain_loss: 1.0990\n",
      "Iteration: 210 of 277\ttrain_loss: 1.0641\n",
      "Iteration: 212 of 277\ttrain_loss: 1.2944\n",
      "Iteration: 214 of 277\ttrain_loss: 1.1625\n",
      "Iteration: 216 of 277\ttrain_loss: 1.2690\n",
      "Iteration: 218 of 277\ttrain_loss: 1.1551\n",
      "Iteration: 220 of 277\ttrain_loss: 1.1854\n",
      "Iteration: 222 of 277\ttrain_loss: 1.3410\n",
      "Iteration: 224 of 277\ttrain_loss: 1.2237\n",
      "Iteration: 226 of 277\ttrain_loss: 1.2433\n",
      "Iteration: 228 of 277\ttrain_loss: 1.2835\n",
      "Iteration: 230 of 277\ttrain_loss: 1.3156\n",
      "Iteration: 232 of 277\ttrain_loss: 1.1993\n",
      "Iteration: 234 of 277\ttrain_loss: 1.2119\n",
      "Iteration: 236 of 277\ttrain_loss: 1.2756\n",
      "Iteration: 238 of 277\ttrain_loss: 1.1734\n",
      "Iteration: 240 of 277\ttrain_loss: 1.1817\n",
      "Iteration: 242 of 277\ttrain_loss: 1.1721\n",
      "Iteration: 244 of 277\ttrain_loss: 1.1569\n",
      "Iteration: 246 of 277\ttrain_loss: 1.1018\n",
      "Iteration: 248 of 277\ttrain_loss: 1.2244\n",
      "Iteration: 250 of 277\ttrain_loss: 1.2506\n",
      "Iteration: 252 of 277\ttrain_loss: 1.0965\n",
      "Iteration: 254 of 277\ttrain_loss: 1.1447\n",
      "Iteration: 256 of 277\ttrain_loss: 1.1610\n",
      "Iteration: 258 of 277\ttrain_loss: 1.4646\n",
      "Iteration: 260 of 277\ttrain_loss: 1.2038\n",
      "Iteration: 262 of 277\ttrain_loss: 1.1491\n",
      "Iteration: 264 of 277\ttrain_loss: 1.2510\n",
      "Iteration: 266 of 277\ttrain_loss: 1.3581\n",
      "Iteration: 268 of 277\ttrain_loss: 1.1001\n",
      "Iteration: 270 of 277\ttrain_loss: 1.3084\n",
      "Iteration: 272 of 277\ttrain_loss: 1.1579\n",
      "Iteration: 274 of 277\ttrain_loss: 1.2210\n",
      "Iteration: 276 of 277\ttrain_loss: 1.2132\n",
      "Iteration: 277 of 277\ttrain_loss: 1.0694\n",
      "Average Score for this Epoch: 1.1937230825424194\n",
      "Eval_loss: 2.5286192893981934\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 26 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.3897\n",
      "Iteration: 2 of 277\ttrain_loss: 1.2058\n",
      "Iteration: 4 of 277\ttrain_loss: 1.1544\n",
      "Iteration: 6 of 277\ttrain_loss: 1.2491\n",
      "Iteration: 8 of 277\ttrain_loss: 1.1974\n",
      "Iteration: 10 of 277\ttrain_loss: 1.1193\n",
      "Iteration: 12 of 277\ttrain_loss: 1.0062\n",
      "Iteration: 14 of 277\ttrain_loss: 1.2186\n",
      "Iteration: 16 of 277\ttrain_loss: 1.1851\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1191\n",
      "Iteration: 20 of 277\ttrain_loss: 1.1781\n",
      "Iteration: 22 of 277\ttrain_loss: 1.1504\n",
      "Iteration: 24 of 277\ttrain_loss: 1.1073\n",
      "Iteration: 26 of 277\ttrain_loss: 1.1231\n",
      "Iteration: 28 of 277\ttrain_loss: 1.0982\n",
      "Iteration: 30 of 277\ttrain_loss: 1.3345\n",
      "Iteration: 32 of 277\ttrain_loss: 1.1602\n",
      "Iteration: 34 of 277\ttrain_loss: 1.1149\n",
      "Iteration: 36 of 277\ttrain_loss: 1.1137\n",
      "Iteration: 38 of 277\ttrain_loss: 1.0568\n",
      "Iteration: 40 of 277\ttrain_loss: 1.1647\n",
      "Iteration: 42 of 277\ttrain_loss: 1.1872\n",
      "Iteration: 44 of 277\ttrain_loss: 1.3324\n",
      "Iteration: 46 of 277\ttrain_loss: 1.2070\n",
      "Iteration: 48 of 277\ttrain_loss: 1.0419\n",
      "Iteration: 50 of 277\ttrain_loss: 1.2138\n",
      "Iteration: 52 of 277\ttrain_loss: 1.2190\n",
      "Iteration: 54 of 277\ttrain_loss: 1.1041\n",
      "Iteration: 56 of 277\ttrain_loss: 1.0337\n",
      "Iteration: 58 of 277\ttrain_loss: 1.1584\n",
      "Iteration: 60 of 277\ttrain_loss: 1.2248\n",
      "Iteration: 62 of 277\ttrain_loss: 1.2848\n",
      "Iteration: 64 of 277\ttrain_loss: 1.0663\n",
      "Iteration: 66 of 277\ttrain_loss: 1.1563\n",
      "Iteration: 68 of 277\ttrain_loss: 1.2030\n",
      "Iteration: 70 of 277\ttrain_loss: 1.0958\n",
      "Iteration: 72 of 277\ttrain_loss: 1.2369\n",
      "Iteration: 74 of 277\ttrain_loss: 1.0983\n",
      "Iteration: 76 of 277\ttrain_loss: 1.2159\n",
      "Iteration: 78 of 277\ttrain_loss: 1.2112\n",
      "Iteration: 80 of 277\ttrain_loss: 1.2093\n",
      "Iteration: 82 of 277\ttrain_loss: 1.1733\n",
      "Iteration: 84 of 277\ttrain_loss: 1.1011\n",
      "Iteration: 86 of 277\ttrain_loss: 1.2736\n",
      "Iteration: 88 of 277\ttrain_loss: 1.1021\n",
      "Iteration: 90 of 277\ttrain_loss: 1.2224\n",
      "Iteration: 92 of 277\ttrain_loss: 1.4458\n",
      "Iteration: 94 of 277\ttrain_loss: 1.2955\n",
      "Iteration: 96 of 277\ttrain_loss: 1.0928\n",
      "Iteration: 98 of 277\ttrain_loss: 1.2154\n",
      "Iteration: 100 of 277\ttrain_loss: 1.2219\n",
      "Iteration: 102 of 277\ttrain_loss: 1.1730\n",
      "Iteration: 104 of 277\ttrain_loss: 1.4130\n",
      "Iteration: 106 of 277\ttrain_loss: 1.2800\n",
      "Iteration: 108 of 277\ttrain_loss: 1.4248\n",
      "Iteration: 110 of 277\ttrain_loss: 1.2024\n",
      "Iteration: 112 of 277\ttrain_loss: 1.0999\n",
      "Iteration: 114 of 277\ttrain_loss: 1.0262\n",
      "Iteration: 116 of 277\ttrain_loss: 1.2657\n",
      "Iteration: 118 of 277\ttrain_loss: 1.2426\n",
      "Iteration: 120 of 277\ttrain_loss: 0.9954\n",
      "Iteration: 122 of 277\ttrain_loss: 1.4103\n",
      "Iteration: 124 of 277\ttrain_loss: 1.0737\n",
      "Iteration: 126 of 277\ttrain_loss: 1.2864\n",
      "Iteration: 128 of 277\ttrain_loss: 1.2647\n",
      "Iteration: 130 of 277\ttrain_loss: 1.2639\n",
      "Iteration: 132 of 277\ttrain_loss: 1.2441\n",
      "Iteration: 134 of 277\ttrain_loss: 1.2195\n",
      "Iteration: 136 of 277\ttrain_loss: 1.0972\n",
      "Iteration: 138 of 277\ttrain_loss: 1.2018\n",
      "Iteration: 140 of 277\ttrain_loss: 1.2270\n",
      "Iteration: 142 of 277\ttrain_loss: 1.1452\n",
      "Iteration: 144 of 277\ttrain_loss: 1.2129\n",
      "Iteration: 146 of 277\ttrain_loss: 1.1300\n",
      "Iteration: 148 of 277\ttrain_loss: 1.1422\n",
      "Iteration: 150 of 277\ttrain_loss: 1.1835\n",
      "Iteration: 152 of 277\ttrain_loss: 1.2463\n",
      "Iteration: 154 of 277\ttrain_loss: 1.1392\n",
      "Iteration: 156 of 277\ttrain_loss: 1.2393\n",
      "Iteration: 158 of 277\ttrain_loss: 1.1538\n",
      "Iteration: 160 of 277\ttrain_loss: 1.1549\n",
      "Iteration: 162 of 277\ttrain_loss: 1.1645\n",
      "Iteration: 164 of 277\ttrain_loss: 1.1410\n",
      "Iteration: 166 of 277\ttrain_loss: 1.1170\n",
      "Iteration: 168 of 277\ttrain_loss: 1.1989\n",
      "Iteration: 170 of 277\ttrain_loss: 1.4741\n",
      "Iteration: 172 of 277\ttrain_loss: 1.2515\n",
      "Iteration: 174 of 277\ttrain_loss: 1.4379\n",
      "Iteration: 176 of 277\ttrain_loss: 1.2258\n",
      "Iteration: 178 of 277\ttrain_loss: 1.6284\n",
      "Iteration: 180 of 277\ttrain_loss: 1.2459\n",
      "Iteration: 182 of 277\ttrain_loss: 1.2480\n",
      "Iteration: 184 of 277\ttrain_loss: 1.1215\n",
      "Iteration: 186 of 277\ttrain_loss: 1.2543\n",
      "Iteration: 188 of 277\ttrain_loss: 1.2442\n",
      "Iteration: 190 of 277\ttrain_loss: 1.0639\n",
      "Iteration: 192 of 277\ttrain_loss: 1.2270\n",
      "Iteration: 194 of 277\ttrain_loss: 1.3023\n",
      "Iteration: 196 of 277\ttrain_loss: 1.1482\n",
      "Iteration: 198 of 277\ttrain_loss: 1.2319\n",
      "Iteration: 200 of 277\ttrain_loss: 1.2047\n",
      "Iteration: 202 of 277\ttrain_loss: 1.2457\n",
      "Iteration: 204 of 277\ttrain_loss: 1.3475\n",
      "Iteration: 206 of 277\ttrain_loss: 1.2693\n",
      "Iteration: 208 of 277\ttrain_loss: 1.4538\n",
      "Iteration: 210 of 277\ttrain_loss: 1.1951\n",
      "Iteration: 212 of 277\ttrain_loss: 1.2109\n",
      "Iteration: 214 of 277\ttrain_loss: 1.1546\n",
      "Iteration: 216 of 277\ttrain_loss: 1.5586\n",
      "Iteration: 218 of 277\ttrain_loss: 1.4209\n",
      "Iteration: 220 of 277\ttrain_loss: 1.2999\n",
      "Iteration: 222 of 277\ttrain_loss: 1.3400\n",
      "Iteration: 224 of 277\ttrain_loss: 1.2200\n",
      "Iteration: 226 of 277\ttrain_loss: 1.4418\n",
      "Iteration: 228 of 277\ttrain_loss: 1.2875\n",
      "Iteration: 230 of 277\ttrain_loss: 1.1623\n",
      "Iteration: 232 of 277\ttrain_loss: 1.3902\n",
      "Iteration: 234 of 277\ttrain_loss: 1.2853\n",
      "Iteration: 236 of 277\ttrain_loss: 1.2583\n",
      "Iteration: 238 of 277\ttrain_loss: 1.2916\n",
      "Iteration: 240 of 277\ttrain_loss: 1.3843\n",
      "Iteration: 242 of 277\ttrain_loss: 1.3037\n",
      "Iteration: 244 of 277\ttrain_loss: 1.1810\n",
      "Iteration: 246 of 277\ttrain_loss: 1.0350\n",
      "Iteration: 248 of 277\ttrain_loss: 1.3193\n",
      "Iteration: 250 of 277\ttrain_loss: 1.4218\n",
      "Iteration: 252 of 277\ttrain_loss: 1.3199\n",
      "Iteration: 254 of 277\ttrain_loss: 1.5620\n",
      "Iteration: 256 of 277\ttrain_loss: 2.3338\n",
      "Iteration: 258 of 277\ttrain_loss: 1.4798\n",
      "Iteration: 260 of 277\ttrain_loss: 1.2693\n",
      "Iteration: 262 of 277\ttrain_loss: 1.3982\n",
      "Iteration: 264 of 277\ttrain_loss: 1.1251\n",
      "Iteration: 266 of 277\ttrain_loss: 1.2359\n",
      "Iteration: 268 of 277\ttrain_loss: 1.1904\n",
      "Iteration: 270 of 277\ttrain_loss: 1.1798\n",
      "Iteration: 272 of 277\ttrain_loss: 1.3179\n",
      "Iteration: 274 of 277\ttrain_loss: 1.2485\n",
      "Iteration: 276 of 277\ttrain_loss: 1.4107\n",
      "Iteration: 277 of 277\ttrain_loss: 1.4589\n",
      "Average Score for this Epoch: 1.227773666381836\n",
      "Eval_loss: 2.5498223304748535\n",
      "\n",
      "-------------------- Epoch 27 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.1812\n",
      "Iteration: 2 of 277\ttrain_loss: 1.2266\n",
      "Iteration: 4 of 277\ttrain_loss: 1.1623\n",
      "Iteration: 6 of 277\ttrain_loss: 1.2491\n",
      "Iteration: 8 of 277\ttrain_loss: 1.2114\n",
      "Iteration: 10 of 277\ttrain_loss: 1.0351\n",
      "Iteration: 12 of 277\ttrain_loss: 1.1319\n",
      "Iteration: 14 of 277\ttrain_loss: 1.3065\n",
      "Iteration: 16 of 277\ttrain_loss: 1.3512\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1767\n",
      "Iteration: 20 of 277\ttrain_loss: 1.1813\n",
      "Iteration: 22 of 277\ttrain_loss: 1.1824\n",
      "Iteration: 24 of 277\ttrain_loss: 1.0972\n",
      "Iteration: 26 of 277\ttrain_loss: 1.1328\n",
      "Iteration: 28 of 277\ttrain_loss: 1.2057\n",
      "Iteration: 30 of 277\ttrain_loss: 1.3403\n",
      "Iteration: 32 of 277\ttrain_loss: 1.3455\n",
      "Iteration: 34 of 277\ttrain_loss: 1.1577\n",
      "Iteration: 36 of 277\ttrain_loss: 1.2540\n",
      "Iteration: 38 of 277\ttrain_loss: 1.1098\n",
      "Iteration: 40 of 277\ttrain_loss: 1.3466\n",
      "Iteration: 42 of 277\ttrain_loss: 1.3596\n",
      "Iteration: 44 of 277\ttrain_loss: 1.3868\n",
      "Iteration: 46 of 277\ttrain_loss: 1.1654\n",
      "Iteration: 48 of 277\ttrain_loss: 1.1092\n",
      "Iteration: 50 of 277\ttrain_loss: 1.0895\n",
      "Iteration: 52 of 277\ttrain_loss: 1.1400\n",
      "Iteration: 54 of 277\ttrain_loss: 1.0900\n",
      "Iteration: 56 of 277\ttrain_loss: 1.3361\n",
      "Iteration: 58 of 277\ttrain_loss: 1.2520\n",
      "Iteration: 60 of 277\ttrain_loss: 1.2601\n",
      "Iteration: 62 of 277\ttrain_loss: 1.1125\n",
      "Iteration: 64 of 277\ttrain_loss: 1.2432\n",
      "Iteration: 66 of 277\ttrain_loss: 1.1115\n",
      "Iteration: 68 of 277\ttrain_loss: 1.2030\n",
      "Iteration: 70 of 277\ttrain_loss: 1.4070\n",
      "Iteration: 72 of 277\ttrain_loss: 1.2530\n",
      "Iteration: 74 of 277\ttrain_loss: 1.2347\n",
      "Iteration: 76 of 277\ttrain_loss: 1.1801\n",
      "Iteration: 78 of 277\ttrain_loss: 1.1369\n",
      "Iteration: 80 of 277\ttrain_loss: 1.5261\n",
      "Iteration: 82 of 277\ttrain_loss: 1.2566\n",
      "Iteration: 84 of 277\ttrain_loss: 1.3356\n",
      "Iteration: 86 of 277\ttrain_loss: 1.1692\n",
      "Iteration: 88 of 277\ttrain_loss: 1.1165\n",
      "Iteration: 90 of 277\ttrain_loss: 1.2604\n",
      "Iteration: 92 of 277\ttrain_loss: 1.2945\n",
      "Iteration: 94 of 277\ttrain_loss: 1.2027\n",
      "Iteration: 96 of 277\ttrain_loss: 1.1684\n",
      "Iteration: 98 of 277\ttrain_loss: 1.3330\n",
      "Iteration: 100 of 277\ttrain_loss: 1.1758\n",
      "Iteration: 102 of 277\ttrain_loss: 1.2820\n",
      "Iteration: 104 of 277\ttrain_loss: 1.4657\n",
      "Iteration: 106 of 277\ttrain_loss: 1.2301\n",
      "Iteration: 108 of 277\ttrain_loss: 1.2204\n",
      "Iteration: 110 of 277\ttrain_loss: 1.2110\n",
      "Iteration: 112 of 277\ttrain_loss: 1.2461\n",
      "Iteration: 114 of 277\ttrain_loss: 1.3352\n",
      "Iteration: 116 of 277\ttrain_loss: 1.2795\n",
      "Iteration: 118 of 277\ttrain_loss: 1.3913\n",
      "Iteration: 120 of 277\ttrain_loss: 1.3116\n",
      "Iteration: 122 of 277\ttrain_loss: 1.2870\n",
      "Iteration: 124 of 277\ttrain_loss: 1.3043\n",
      "Iteration: 126 of 277\ttrain_loss: 1.2875\n",
      "Iteration: 128 of 277\ttrain_loss: 1.3521\n",
      "Iteration: 130 of 277\ttrain_loss: 1.2671\n",
      "Iteration: 132 of 277\ttrain_loss: 1.2749\n",
      "Iteration: 134 of 277\ttrain_loss: 1.2429\n",
      "Iteration: 136 of 277\ttrain_loss: 1.3261\n",
      "Iteration: 138 of 277\ttrain_loss: 1.3453\n",
      "Iteration: 140 of 277\ttrain_loss: 1.1273\n",
      "Iteration: 142 of 277\ttrain_loss: 1.1951\n",
      "Iteration: 144 of 277\ttrain_loss: 1.3638\n",
      "Iteration: 146 of 277\ttrain_loss: 1.2157\n",
      "Iteration: 148 of 277\ttrain_loss: 1.2614\n",
      "Iteration: 150 of 277\ttrain_loss: 1.3038\n",
      "Iteration: 152 of 277\ttrain_loss: 1.3121\n",
      "Iteration: 154 of 277\ttrain_loss: 1.3078\n",
      "Iteration: 156 of 277\ttrain_loss: 1.2041\n",
      "Iteration: 158 of 277\ttrain_loss: 1.6231\n",
      "Iteration: 160 of 277\ttrain_loss: 1.3408\n",
      "Iteration: 162 of 277\ttrain_loss: 1.2044\n",
      "Iteration: 164 of 277\ttrain_loss: 1.3500\n",
      "Iteration: 166 of 277\ttrain_loss: 1.1799\n",
      "Iteration: 168 of 277\ttrain_loss: 1.3527\n",
      "Iteration: 170 of 277\ttrain_loss: 1.3366\n",
      "Iteration: 172 of 277\ttrain_loss: 1.2267\n",
      "Iteration: 174 of 277\ttrain_loss: 1.3480\n",
      "Iteration: 176 of 277\ttrain_loss: 1.2181\n",
      "Iteration: 178 of 277\ttrain_loss: 1.3278\n",
      "Iteration: 180 of 277\ttrain_loss: 1.3347\n",
      "Iteration: 182 of 277\ttrain_loss: 1.4397\n",
      "Iteration: 184 of 277\ttrain_loss: 1.2004\n",
      "Iteration: 186 of 277\ttrain_loss: 1.3957\n",
      "Iteration: 188 of 277\ttrain_loss: 1.3927\n",
      "Iteration: 190 of 277\ttrain_loss: 1.3791\n",
      "Iteration: 192 of 277\ttrain_loss: 1.3835\n",
      "Iteration: 194 of 277\ttrain_loss: 1.2178\n",
      "Iteration: 196 of 277\ttrain_loss: 1.1513\n",
      "Iteration: 198 of 277\ttrain_loss: 1.3048\n",
      "Iteration: 200 of 277\ttrain_loss: 1.4655\n",
      "Iteration: 202 of 277\ttrain_loss: 1.2794\n",
      "Iteration: 204 of 277\ttrain_loss: 1.3117\n",
      "Iteration: 206 of 277\ttrain_loss: 1.3257\n",
      "Iteration: 208 of 277\ttrain_loss: 1.4114\n",
      "Iteration: 210 of 277\ttrain_loss: 1.4485\n",
      "Iteration: 212 of 277\ttrain_loss: 1.3297\n",
      "Iteration: 214 of 277\ttrain_loss: 1.2514\n",
      "Iteration: 216 of 277\ttrain_loss: 1.2742\n",
      "Iteration: 218 of 277\ttrain_loss: 1.4847\n",
      "Iteration: 220 of 277\ttrain_loss: 1.3540\n",
      "Iteration: 222 of 277\ttrain_loss: 1.5172\n",
      "Iteration: 224 of 277\ttrain_loss: 1.5240\n",
      "Iteration: 226 of 277\ttrain_loss: 1.2841\n",
      "Iteration: 228 of 277\ttrain_loss: 1.2157\n",
      "Iteration: 230 of 277\ttrain_loss: 1.3450\n",
      "Iteration: 232 of 277\ttrain_loss: 1.3131\n",
      "Iteration: 234 of 277\ttrain_loss: 1.2455\n",
      "Iteration: 236 of 277\ttrain_loss: 1.1246\n",
      "Iteration: 238 of 277\ttrain_loss: 1.3628\n",
      "Iteration: 240 of 277\ttrain_loss: 1.2836\n",
      "Iteration: 242 of 277\ttrain_loss: 1.3787\n",
      "Iteration: 244 of 277\ttrain_loss: 1.3121\n",
      "Iteration: 246 of 277\ttrain_loss: 1.3976\n",
      "Iteration: 248 of 277\ttrain_loss: 1.3866\n",
      "Iteration: 250 of 277\ttrain_loss: 1.3988\n",
      "Iteration: 252 of 277\ttrain_loss: 1.2376\n",
      "Iteration: 254 of 277\ttrain_loss: 1.3996\n",
      "Iteration: 256 of 277\ttrain_loss: 1.5638\n",
      "Iteration: 258 of 277\ttrain_loss: 1.2496\n",
      "Iteration: 260 of 277\ttrain_loss: 1.3398\n",
      "Iteration: 262 of 277\ttrain_loss: 1.4055\n",
      "Iteration: 264 of 277\ttrain_loss: 1.3396\n",
      "Iteration: 266 of 277\ttrain_loss: 1.3938\n",
      "Iteration: 268 of 277\ttrain_loss: 1.3550\n",
      "Iteration: 270 of 277\ttrain_loss: 1.3428\n",
      "Iteration: 272 of 277\ttrain_loss: 1.4165\n",
      "Iteration: 274 of 277\ttrain_loss: 1.3074\n",
      "Iteration: 276 of 277\ttrain_loss: 1.1654\n",
      "Iteration: 277 of 277\ttrain_loss: 1.2823\n",
      "Average Score for this Epoch: 1.2895722389221191\n",
      "Eval_loss: 2.6077964305877686\n",
      "\n",
      "-------------------- Epoch 28 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.3199\n",
      "Iteration: 2 of 277\ttrain_loss: 1.2387\n",
      "Iteration: 4 of 277\ttrain_loss: 1.1472\n",
      "Iteration: 6 of 277\ttrain_loss: 1.2310\n",
      "Iteration: 8 of 277\ttrain_loss: 1.2461\n",
      "Iteration: 10 of 277\ttrain_loss: 1.1416\n",
      "Iteration: 12 of 277\ttrain_loss: 1.2096\n",
      "Iteration: 14 of 277\ttrain_loss: 1.2263\n",
      "Iteration: 16 of 277\ttrain_loss: 1.2504\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1988\n",
      "Iteration: 20 of 277\ttrain_loss: 1.2038\n",
      "Iteration: 22 of 277\ttrain_loss: 1.2017\n",
      "Iteration: 24 of 277\ttrain_loss: 1.0944\n",
      "Iteration: 26 of 277\ttrain_loss: 1.2302\n",
      "Iteration: 28 of 277\ttrain_loss: 1.2186\n",
      "Iteration: 30 of 277\ttrain_loss: 1.2966\n",
      "Iteration: 32 of 277\ttrain_loss: 1.1258\n",
      "Iteration: 34 of 277\ttrain_loss: 1.1971\n",
      "Iteration: 36 of 277\ttrain_loss: 1.2198\n",
      "Iteration: 38 of 277\ttrain_loss: 1.2767\n",
      "Iteration: 40 of 277\ttrain_loss: 1.2751\n",
      "Iteration: 42 of 277\ttrain_loss: 1.1786\n",
      "Iteration: 44 of 277\ttrain_loss: 1.2258\n",
      "Iteration: 46 of 277\ttrain_loss: 1.2628\n",
      "Iteration: 48 of 277\ttrain_loss: 1.2939\n",
      "Iteration: 50 of 277\ttrain_loss: 1.1345\n",
      "Iteration: 52 of 277\ttrain_loss: 1.2186\n",
      "Iteration: 54 of 277\ttrain_loss: 1.1009\n",
      "Iteration: 56 of 277\ttrain_loss: 1.1514\n",
      "Iteration: 58 of 277\ttrain_loss: 1.1768\n",
      "Iteration: 60 of 277\ttrain_loss: 1.3587\n",
      "Iteration: 62 of 277\ttrain_loss: 1.1087\n",
      "Iteration: 64 of 277\ttrain_loss: 1.1291\n",
      "Iteration: 66 of 277\ttrain_loss: 1.2016\n",
      "Iteration: 68 of 277\ttrain_loss: 1.1001\n",
      "Iteration: 70 of 277\ttrain_loss: 1.2471\n",
      "Iteration: 72 of 277\ttrain_loss: 1.1789\n",
      "Iteration: 74 of 277\ttrain_loss: 1.1464\n",
      "Iteration: 76 of 277\ttrain_loss: 1.1889\n",
      "Iteration: 78 of 277\ttrain_loss: 1.2424\n",
      "Iteration: 80 of 277\ttrain_loss: 1.2598\n",
      "Iteration: 82 of 277\ttrain_loss: 1.1802\n",
      "Iteration: 84 of 277\ttrain_loss: 1.2567\n",
      "Iteration: 86 of 277\ttrain_loss: 1.2182\n",
      "Iteration: 88 of 277\ttrain_loss: 1.1367\n",
      "Iteration: 90 of 277\ttrain_loss: 1.4523\n",
      "Iteration: 92 of 277\ttrain_loss: 1.2317\n",
      "Iteration: 94 of 277\ttrain_loss: 1.1423\n",
      "Iteration: 96 of 277\ttrain_loss: 1.1499\n",
      "Iteration: 98 of 277\ttrain_loss: 1.1648\n",
      "Iteration: 100 of 277\ttrain_loss: 1.2234\n",
      "Iteration: 102 of 277\ttrain_loss: 1.1324\n",
      "Iteration: 104 of 277\ttrain_loss: 1.2302\n",
      "Iteration: 106 of 277\ttrain_loss: 1.2800\n",
      "Iteration: 108 of 277\ttrain_loss: 2.4326\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0955\n",
      "Iteration: 112 of 277\ttrain_loss: 1.5594\n",
      "Iteration: 114 of 277\ttrain_loss: 1.2711\n",
      "Iteration: 116 of 277\ttrain_loss: 1.2221\n",
      "Iteration: 118 of 277\ttrain_loss: 1.1869\n",
      "Iteration: 120 of 277\ttrain_loss: 1.2250\n",
      "Iteration: 122 of 277\ttrain_loss: 1.2633\n",
      "Iteration: 124 of 277\ttrain_loss: 1.0781\n",
      "Iteration: 126 of 277\ttrain_loss: 1.1773\n",
      "Iteration: 128 of 277\ttrain_loss: 1.3442\n",
      "Iteration: 130 of 277\ttrain_loss: 1.4982\n",
      "Iteration: 132 of 277\ttrain_loss: 1.2073\n",
      "Iteration: 134 of 277\ttrain_loss: 1.2671\n",
      "Iteration: 136 of 277\ttrain_loss: 1.3026\n",
      "Iteration: 138 of 277\ttrain_loss: 1.2256\n",
      "Iteration: 140 of 277\ttrain_loss: 1.2770\n",
      "Iteration: 142 of 277\ttrain_loss: 1.2984\n",
      "Iteration: 144 of 277\ttrain_loss: 1.1705\n",
      "Iteration: 146 of 277\ttrain_loss: 1.0827\n",
      "Iteration: 148 of 277\ttrain_loss: 1.2966\n",
      "Iteration: 150 of 277\ttrain_loss: 1.2751\n",
      "Iteration: 152 of 277\ttrain_loss: 1.3785\n",
      "Iteration: 154 of 277\ttrain_loss: 1.1846\n",
      "Iteration: 156 of 277\ttrain_loss: 1.1318\n",
      "Iteration: 158 of 277\ttrain_loss: 1.1791\n",
      "Iteration: 160 of 277\ttrain_loss: 1.3525\n",
      "Iteration: 162 of 277\ttrain_loss: 1.2164\n",
      "Iteration: 164 of 277\ttrain_loss: 1.4901\n",
      "Iteration: 166 of 277\ttrain_loss: 1.2461\n",
      "Iteration: 168 of 277\ttrain_loss: 1.1994\n",
      "Iteration: 170 of 277\ttrain_loss: 1.2928\n",
      "Iteration: 172 of 277\ttrain_loss: 1.3159\n",
      "Iteration: 174 of 277\ttrain_loss: 1.2937\n",
      "Iteration: 176 of 277\ttrain_loss: 1.3097\n",
      "Iteration: 178 of 277\ttrain_loss: 1.2481\n",
      "Iteration: 180 of 277\ttrain_loss: 1.1961\n",
      "Iteration: 182 of 277\ttrain_loss: 1.2435\n",
      "Iteration: 184 of 277\ttrain_loss: 1.3657\n",
      "Iteration: 186 of 277\ttrain_loss: 1.1195\n",
      "Iteration: 188 of 277\ttrain_loss: 1.2441\n",
      "Iteration: 190 of 277\ttrain_loss: 1.1493\n",
      "Iteration: 192 of 277\ttrain_loss: 1.3026\n",
      "Iteration: 194 of 277\ttrain_loss: 1.3957\n",
      "Iteration: 196 of 277\ttrain_loss: 1.1824\n",
      "Iteration: 198 of 277\ttrain_loss: 1.2425\n",
      "Iteration: 200 of 277\ttrain_loss: 1.1959\n",
      "Iteration: 202 of 277\ttrain_loss: 1.1201\n",
      "Iteration: 204 of 277\ttrain_loss: 1.1589\n",
      "Iteration: 206 of 277\ttrain_loss: 1.2434\n",
      "Iteration: 208 of 277\ttrain_loss: 1.1385\n",
      "Iteration: 210 of 277\ttrain_loss: 1.1681\n",
      "Iteration: 212 of 277\ttrain_loss: 1.2854\n",
      "Iteration: 214 of 277\ttrain_loss: 1.1253\n",
      "Iteration: 216 of 277\ttrain_loss: 1.3074\n",
      "Iteration: 218 of 277\ttrain_loss: 1.3860\n",
      "Iteration: 220 of 277\ttrain_loss: 1.3735\n",
      "Iteration: 222 of 277\ttrain_loss: 1.2627\n",
      "Iteration: 224 of 277\ttrain_loss: 1.2754\n",
      "Iteration: 226 of 277\ttrain_loss: 1.3385\n",
      "Iteration: 228 of 277\ttrain_loss: 1.1812\n",
      "Iteration: 230 of 277\ttrain_loss: 1.2501\n",
      "Iteration: 232 of 277\ttrain_loss: 1.2295\n",
      "Iteration: 234 of 277\ttrain_loss: 1.2439\n",
      "Iteration: 236 of 277\ttrain_loss: 1.2225\n",
      "Iteration: 238 of 277\ttrain_loss: 1.2434\n",
      "Iteration: 240 of 277\ttrain_loss: 1.1490\n",
      "Iteration: 242 of 277\ttrain_loss: 1.4252\n",
      "Iteration: 244 of 277\ttrain_loss: 1.1937\n",
      "Iteration: 246 of 277\ttrain_loss: 1.3529\n",
      "Iteration: 248 of 277\ttrain_loss: 1.4757\n",
      "Iteration: 250 of 277\ttrain_loss: 1.6001\n",
      "Iteration: 252 of 277\ttrain_loss: 1.4303\n",
      "Iteration: 254 of 277\ttrain_loss: 1.3189\n",
      "Iteration: 256 of 277\ttrain_loss: 1.3548\n",
      "Iteration: 258 of 277\ttrain_loss: 1.2088\n",
      "Iteration: 260 of 277\ttrain_loss: 1.4349\n",
      "Iteration: 262 of 277\ttrain_loss: 1.3174\n",
      "Iteration: 264 of 277\ttrain_loss: 1.4273\n",
      "Iteration: 266 of 277\ttrain_loss: 1.1092\n",
      "Iteration: 268 of 277\ttrain_loss: 1.2330\n",
      "Iteration: 270 of 277\ttrain_loss: 1.2764\n",
      "Iteration: 272 of 277\ttrain_loss: 1.2025\n",
      "Iteration: 274 of 277\ttrain_loss: 1.4335\n",
      "Iteration: 276 of 277\ttrain_loss: 1.2082\n",
      "Iteration: 277 of 277\ttrain_loss: 1.1429\n",
      "Average Score for this Epoch: 1.25504469871521\n",
      "Eval_loss: 2.5569255352020264\n",
      "\n",
      "-------------------- Epoch 29 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.2973\n",
      "Iteration: 2 of 277\ttrain_loss: 1.0968\n",
      "Iteration: 4 of 277\ttrain_loss: 1.1059\n",
      "Iteration: 6 of 277\ttrain_loss: 1.0617\n",
      "Iteration: 8 of 277\ttrain_loss: 1.1604\n",
      "Iteration: 10 of 277\ttrain_loss: 1.2641\n",
      "Iteration: 12 of 277\ttrain_loss: 1.0567\n",
      "Iteration: 14 of 277\ttrain_loss: 1.2229\n",
      "Iteration: 16 of 277\ttrain_loss: 1.1161\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1840\n",
      "Iteration: 20 of 277\ttrain_loss: 1.2466\n",
      "Iteration: 22 of 277\ttrain_loss: 1.1116\n",
      "Iteration: 24 of 277\ttrain_loss: 1.2661\n",
      "Iteration: 26 of 277\ttrain_loss: 1.0683\n",
      "Iteration: 28 of 277\ttrain_loss: 1.0460\n",
      "Iteration: 30 of 277\ttrain_loss: 1.0760\n",
      "Iteration: 32 of 277\ttrain_loss: 1.2084\n",
      "Iteration: 34 of 277\ttrain_loss: 1.1347\n",
      "Iteration: 36 of 277\ttrain_loss: 1.1682\n",
      "Iteration: 38 of 277\ttrain_loss: 1.0936\n",
      "Iteration: 40 of 277\ttrain_loss: 1.1358\n",
      "Iteration: 42 of 277\ttrain_loss: 1.1352\n",
      "Iteration: 44 of 277\ttrain_loss: 1.0588\n",
      "Iteration: 46 of 277\ttrain_loss: 1.0168\n",
      "Iteration: 48 of 277\ttrain_loss: 1.2445\n",
      "Iteration: 50 of 277\ttrain_loss: 1.0990\n",
      "Iteration: 52 of 277\ttrain_loss: 1.1284\n",
      "Iteration: 54 of 277\ttrain_loss: 1.2130\n",
      "Iteration: 56 of 277\ttrain_loss: 0.9876\n",
      "Iteration: 58 of 277\ttrain_loss: 1.0693\n",
      "Iteration: 60 of 277\ttrain_loss: 1.0988\n",
      "Iteration: 62 of 277\ttrain_loss: 1.1526\n",
      "Iteration: 64 of 277\ttrain_loss: 1.0628\n",
      "Iteration: 66 of 277\ttrain_loss: 1.1038\n",
      "Iteration: 68 of 277\ttrain_loss: 1.1943\n",
      "Iteration: 70 of 277\ttrain_loss: 1.2808\n",
      "Iteration: 72 of 277\ttrain_loss: 1.1565\n",
      "Iteration: 74 of 277\ttrain_loss: 1.1333\n",
      "Iteration: 76 of 277\ttrain_loss: 1.1698\n",
      "Iteration: 78 of 277\ttrain_loss: 1.1284\n",
      "Iteration: 80 of 277\ttrain_loss: 1.1433\n",
      "Iteration: 82 of 277\ttrain_loss: 1.0509\n",
      "Iteration: 84 of 277\ttrain_loss: 1.1596\n",
      "Iteration: 86 of 277\ttrain_loss: 1.1185\n",
      "Iteration: 88 of 277\ttrain_loss: 1.0940\n",
      "Iteration: 90 of 277\ttrain_loss: 1.0561\n",
      "Iteration: 92 of 277\ttrain_loss: 1.0736\n",
      "Iteration: 94 of 277\ttrain_loss: 1.0099\n",
      "Iteration: 96 of 277\ttrain_loss: 1.1056\n",
      "Iteration: 98 of 277\ttrain_loss: 1.1483\n",
      "Iteration: 100 of 277\ttrain_loss: 0.9829\n",
      "Iteration: 102 of 277\ttrain_loss: 1.2111\n",
      "Iteration: 104 of 277\ttrain_loss: 1.1186\n",
      "Iteration: 106 of 277\ttrain_loss: 1.1502\n",
      "Iteration: 108 of 277\ttrain_loss: 1.1378\n",
      "Iteration: 110 of 277\ttrain_loss: 1.1753\n",
      "Iteration: 112 of 277\ttrain_loss: 1.4431\n",
      "Iteration: 114 of 277\ttrain_loss: 1.0651\n",
      "Iteration: 116 of 277\ttrain_loss: 1.0116\n",
      "Iteration: 118 of 277\ttrain_loss: 1.0832\n",
      "Iteration: 120 of 277\ttrain_loss: 1.1380\n",
      "Iteration: 122 of 277\ttrain_loss: 1.0871\n",
      "Iteration: 124 of 277\ttrain_loss: 1.2176\n",
      "Iteration: 126 of 277\ttrain_loss: 1.0351\n",
      "Iteration: 128 of 277\ttrain_loss: 1.2189\n",
      "Iteration: 130 of 277\ttrain_loss: 1.1661\n",
      "Iteration: 132 of 277\ttrain_loss: 1.0511\n",
      "Iteration: 134 of 277\ttrain_loss: 1.1451\n",
      "Iteration: 136 of 277\ttrain_loss: 1.0537\n",
      "Iteration: 138 of 277\ttrain_loss: 1.1167\n",
      "Iteration: 140 of 277\ttrain_loss: 1.1697\n",
      "Iteration: 142 of 277\ttrain_loss: 1.3425\n",
      "Iteration: 144 of 277\ttrain_loss: 1.1244\n",
      "Iteration: 146 of 277\ttrain_loss: 1.4508\n",
      "Iteration: 148 of 277\ttrain_loss: 1.1403\n",
      "Iteration: 150 of 277\ttrain_loss: 1.0662\n",
      "Iteration: 152 of 277\ttrain_loss: 1.1388\n",
      "Iteration: 154 of 277\ttrain_loss: 1.1081\n",
      "Iteration: 156 of 277\ttrain_loss: 1.1477\n",
      "Iteration: 158 of 277\ttrain_loss: 1.1547\n",
      "Iteration: 160 of 277\ttrain_loss: 1.1599\n",
      "Iteration: 162 of 277\ttrain_loss: 1.0812\n",
      "Iteration: 164 of 277\ttrain_loss: 1.0584\n",
      "Iteration: 166 of 277\ttrain_loss: 1.0621\n",
      "Iteration: 168 of 277\ttrain_loss: 1.0476\n",
      "Iteration: 170 of 277\ttrain_loss: 1.2327\n",
      "Iteration: 172 of 277\ttrain_loss: 1.1074\n",
      "Iteration: 174 of 277\ttrain_loss: 1.0605\n",
      "Iteration: 176 of 277\ttrain_loss: 1.1558\n",
      "Iteration: 178 of 277\ttrain_loss: 1.0902\n",
      "Iteration: 180 of 277\ttrain_loss: 1.0618\n",
      "Iteration: 182 of 277\ttrain_loss: 1.0850\n",
      "Iteration: 184 of 277\ttrain_loss: 1.2161\n",
      "Iteration: 186 of 277\ttrain_loss: 1.0325\n",
      "Iteration: 188 of 277\ttrain_loss: 1.2139\n",
      "Iteration: 190 of 277\ttrain_loss: 1.0893\n",
      "Iteration: 192 of 277\ttrain_loss: 1.2151\n",
      "Iteration: 194 of 277\ttrain_loss: 1.1296\n",
      "Iteration: 196 of 277\ttrain_loss: 1.1081\n",
      "Iteration: 198 of 277\ttrain_loss: 1.2751\n",
      "Iteration: 200 of 277\ttrain_loss: 1.1397\n",
      "Iteration: 202 of 277\ttrain_loss: 1.3826\n",
      "Iteration: 204 of 277\ttrain_loss: 1.0824\n",
      "Iteration: 206 of 277\ttrain_loss: 1.2078\n",
      "Iteration: 208 of 277\ttrain_loss: 1.1488\n",
      "Iteration: 210 of 277\ttrain_loss: 1.1153\n",
      "Iteration: 212 of 277\ttrain_loss: 1.0922\n",
      "Iteration: 214 of 277\ttrain_loss: 1.2263\n",
      "Iteration: 216 of 277\ttrain_loss: 1.0004\n",
      "Iteration: 218 of 277\ttrain_loss: 1.2189\n",
      "Iteration: 220 of 277\ttrain_loss: 1.1822\n",
      "Iteration: 222 of 277\ttrain_loss: 1.0081\n",
      "Iteration: 224 of 277\ttrain_loss: 1.3079\n",
      "Iteration: 226 of 277\ttrain_loss: 1.0944\n",
      "Iteration: 228 of 277\ttrain_loss: 1.1306\n",
      "Iteration: 230 of 277\ttrain_loss: 1.3663\n",
      "Iteration: 232 of 277\ttrain_loss: 1.0928\n",
      "Iteration: 234 of 277\ttrain_loss: 1.1379\n",
      "Iteration: 236 of 277\ttrain_loss: 1.1028\n",
      "Iteration: 238 of 277\ttrain_loss: 1.1695\n",
      "Iteration: 240 of 277\ttrain_loss: 1.1424\n",
      "Iteration: 242 of 277\ttrain_loss: 1.0564\n",
      "Iteration: 244 of 277\ttrain_loss: 1.3090\n",
      "Iteration: 246 of 277\ttrain_loss: 1.1573\n",
      "Iteration: 248 of 277\ttrain_loss: 1.3209\n",
      "Iteration: 250 of 277\ttrain_loss: 1.1782\n",
      "Iteration: 252 of 277\ttrain_loss: 1.1850\n",
      "Iteration: 254 of 277\ttrain_loss: 1.3018\n",
      "Iteration: 256 of 277\ttrain_loss: 1.0660\n",
      "Iteration: 258 of 277\ttrain_loss: 1.4030\n",
      "Iteration: 260 of 277\ttrain_loss: 1.2138\n",
      "Iteration: 262 of 277\ttrain_loss: 1.0444\n",
      "Iteration: 264 of 277\ttrain_loss: 1.1242\n",
      "Iteration: 266 of 277\ttrain_loss: 1.2351\n",
      "Iteration: 268 of 277\ttrain_loss: 1.2344\n",
      "Iteration: 270 of 277\ttrain_loss: 1.1860\n",
      "Iteration: 272 of 277\ttrain_loss: 1.1314\n",
      "Iteration: 274 of 277\ttrain_loss: 1.0328\n",
      "Iteration: 276 of 277\ttrain_loss: 1.0598\n",
      "Iteration: 277 of 277\ttrain_loss: 1.2998\n",
      "Average Score for this Epoch: 1.1546263694763184\n",
      "Eval_loss: 2.551175594329834\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 30 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.0093\n",
      "Iteration: 2 of 277\ttrain_loss: 1.0618\n",
      "Iteration: 4 of 277\ttrain_loss: 1.0050\n",
      "Iteration: 6 of 277\ttrain_loss: 0.9529\n",
      "Iteration: 8 of 277\ttrain_loss: 1.0754\n",
      "Iteration: 10 of 277\ttrain_loss: 0.9749\n",
      "Iteration: 12 of 277\ttrain_loss: 1.2305\n",
      "Iteration: 14 of 277\ttrain_loss: 1.2163\n",
      "Iteration: 16 of 277\ttrain_loss: 1.1260\n",
      "Iteration: 18 of 277\ttrain_loss: 1.0589\n",
      "Iteration: 20 of 277\ttrain_loss: 1.1189\n",
      "Iteration: 22 of 277\ttrain_loss: 1.1892\n",
      "Iteration: 24 of 277\ttrain_loss: 1.1032\n",
      "Iteration: 26 of 277\ttrain_loss: 1.2212\n",
      "Iteration: 28 of 277\ttrain_loss: 1.1019\n",
      "Iteration: 30 of 277\ttrain_loss: 1.0606\n",
      "Iteration: 32 of 277\ttrain_loss: 1.0686\n",
      "Iteration: 34 of 277\ttrain_loss: 1.0840\n",
      "Iteration: 36 of 277\ttrain_loss: 1.1114\n",
      "Iteration: 38 of 277\ttrain_loss: 1.1533\n",
      "Iteration: 40 of 277\ttrain_loss: 1.2280\n",
      "Iteration: 42 of 277\ttrain_loss: 1.0114\n",
      "Iteration: 44 of 277\ttrain_loss: 1.0568\n",
      "Iteration: 46 of 277\ttrain_loss: 1.0674\n",
      "Iteration: 48 of 277\ttrain_loss: 1.0841\n",
      "Iteration: 50 of 277\ttrain_loss: 1.0072\n",
      "Iteration: 52 of 277\ttrain_loss: 1.1121\n",
      "Iteration: 54 of 277\ttrain_loss: 1.1051\n",
      "Iteration: 56 of 277\ttrain_loss: 0.9950\n",
      "Iteration: 58 of 277\ttrain_loss: 1.0752\n",
      "Iteration: 60 of 277\ttrain_loss: 1.1910\n",
      "Iteration: 62 of 277\ttrain_loss: 1.1021\n",
      "Iteration: 64 of 277\ttrain_loss: 1.1051\n",
      "Iteration: 66 of 277\ttrain_loss: 1.0056\n",
      "Iteration: 68 of 277\ttrain_loss: 1.1742\n",
      "Iteration: 70 of 277\ttrain_loss: 1.0680\n",
      "Iteration: 72 of 277\ttrain_loss: 1.1784\n",
      "Iteration: 74 of 277\ttrain_loss: 0.8600\n",
      "Iteration: 76 of 277\ttrain_loss: 1.0562\n",
      "Iteration: 78 of 277\ttrain_loss: 1.1481\n",
      "Iteration: 80 of 277\ttrain_loss: 1.1802\n",
      "Iteration: 82 of 277\ttrain_loss: 1.2259\n",
      "Iteration: 84 of 277\ttrain_loss: 1.0868\n",
      "Iteration: 86 of 277\ttrain_loss: 1.0124\n",
      "Iteration: 88 of 277\ttrain_loss: 1.0100\n",
      "Iteration: 90 of 277\ttrain_loss: 1.0737\n",
      "Iteration: 92 of 277\ttrain_loss: 1.0071\n",
      "Iteration: 94 of 277\ttrain_loss: 1.2074\n",
      "Iteration: 96 of 277\ttrain_loss: 1.1084\n",
      "Iteration: 98 of 277\ttrain_loss: 1.3446\n",
      "Iteration: 100 of 277\ttrain_loss: 1.0173\n",
      "Iteration: 102 of 277\ttrain_loss: 1.0568\n",
      "Iteration: 104 of 277\ttrain_loss: 1.0832\n",
      "Iteration: 106 of 277\ttrain_loss: 1.2404\n",
      "Iteration: 108 of 277\ttrain_loss: 1.0137\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0731\n",
      "Iteration: 112 of 277\ttrain_loss: 1.0969\n",
      "Iteration: 114 of 277\ttrain_loss: 0.9754\n",
      "Iteration: 116 of 277\ttrain_loss: 0.9479\n",
      "Iteration: 118 of 277\ttrain_loss: 0.9603\n",
      "Iteration: 120 of 277\ttrain_loss: 1.0803\n",
      "Iteration: 122 of 277\ttrain_loss: 1.2068\n",
      "Iteration: 124 of 277\ttrain_loss: 1.0963\n",
      "Iteration: 126 of 277\ttrain_loss: 1.1409\n",
      "Iteration: 128 of 277\ttrain_loss: 1.1121\n",
      "Iteration: 130 of 277\ttrain_loss: 1.3051\n",
      "Iteration: 132 of 277\ttrain_loss: 0.9770\n",
      "Iteration: 134 of 277\ttrain_loss: 1.0674\n",
      "Iteration: 136 of 277\ttrain_loss: 1.1873\n",
      "Iteration: 138 of 277\ttrain_loss: 1.1172\n",
      "Iteration: 140 of 277\ttrain_loss: 1.1124\n",
      "Iteration: 142 of 277\ttrain_loss: 1.0633\n",
      "Iteration: 144 of 277\ttrain_loss: 1.0318\n",
      "Iteration: 146 of 277\ttrain_loss: 1.0092\n",
      "Iteration: 148 of 277\ttrain_loss: 1.3185\n",
      "Iteration: 150 of 277\ttrain_loss: 1.2125\n",
      "Iteration: 152 of 277\ttrain_loss: 1.0088\n",
      "Iteration: 154 of 277\ttrain_loss: 1.2095\n",
      "Iteration: 156 of 277\ttrain_loss: 1.0847\n",
      "Iteration: 158 of 277\ttrain_loss: 1.0158\n",
      "Iteration: 160 of 277\ttrain_loss: 1.1872\n",
      "Iteration: 162 of 277\ttrain_loss: 1.0505\n",
      "Iteration: 164 of 277\ttrain_loss: 1.0396\n",
      "Iteration: 166 of 277\ttrain_loss: 1.1115\n",
      "Iteration: 168 of 277\ttrain_loss: 1.0191\n",
      "Iteration: 170 of 277\ttrain_loss: 1.0706\n",
      "Iteration: 172 of 277\ttrain_loss: 0.9662\n",
      "Iteration: 174 of 277\ttrain_loss: 1.2738\n",
      "Iteration: 176 of 277\ttrain_loss: 1.0373\n",
      "Iteration: 178 of 277\ttrain_loss: 1.2415\n",
      "Iteration: 180 of 277\ttrain_loss: 1.0065\n",
      "Iteration: 182 of 277\ttrain_loss: 1.1333\n",
      "Iteration: 184 of 277\ttrain_loss: 1.2380\n",
      "Iteration: 186 of 277\ttrain_loss: 1.1699\n",
      "Iteration: 188 of 277\ttrain_loss: 1.3835\n",
      "Iteration: 190 of 277\ttrain_loss: 1.2871\n",
      "Iteration: 192 of 277\ttrain_loss: 1.0021\n",
      "Iteration: 194 of 277\ttrain_loss: 1.0309\n",
      "Iteration: 196 of 277\ttrain_loss: 1.1237\n",
      "Iteration: 198 of 277\ttrain_loss: 1.0524\n",
      "Iteration: 200 of 277\ttrain_loss: 0.9901\n",
      "Iteration: 202 of 277\ttrain_loss: 1.0784\n",
      "Iteration: 204 of 277\ttrain_loss: 1.1514\n",
      "Iteration: 206 of 277\ttrain_loss: 1.2190\n",
      "Iteration: 208 of 277\ttrain_loss: 1.0586\n",
      "Iteration: 210 of 277\ttrain_loss: 1.1979\n",
      "Iteration: 212 of 277\ttrain_loss: 1.0193\n",
      "Iteration: 214 of 277\ttrain_loss: 1.0571\n",
      "Iteration: 216 of 277\ttrain_loss: 1.1851\n",
      "Iteration: 218 of 277\ttrain_loss: 1.0266\n",
      "Iteration: 220 of 277\ttrain_loss: 1.2529\n",
      "Iteration: 222 of 277\ttrain_loss: 1.1092\n",
      "Iteration: 224 of 277\ttrain_loss: 1.1391\n",
      "Iteration: 226 of 277\ttrain_loss: 1.4087\n",
      "Iteration: 228 of 277\ttrain_loss: 1.2483\n",
      "Iteration: 230 of 277\ttrain_loss: 0.9719\n",
      "Iteration: 232 of 277\ttrain_loss: 1.0570\n",
      "Iteration: 234 of 277\ttrain_loss: 1.1099\n",
      "Iteration: 236 of 277\ttrain_loss: 1.1540\n",
      "Iteration: 238 of 277\ttrain_loss: 1.1220\n",
      "Iteration: 240 of 277\ttrain_loss: 1.0843\n",
      "Iteration: 242 of 277\ttrain_loss: 1.0839\n",
      "Iteration: 244 of 277\ttrain_loss: 1.0111\n",
      "Iteration: 246 of 277\ttrain_loss: 1.1647\n",
      "Iteration: 248 of 277\ttrain_loss: 1.0421\n",
      "Iteration: 250 of 277\ttrain_loss: 1.0589\n",
      "Iteration: 252 of 277\ttrain_loss: 0.9916\n",
      "Iteration: 254 of 277\ttrain_loss: 1.0330\n",
      "Iteration: 256 of 277\ttrain_loss: 1.1088\n",
      "Iteration: 258 of 277\ttrain_loss: 1.0244\n",
      "Iteration: 260 of 277\ttrain_loss: 1.1630\n",
      "Iteration: 262 of 277\ttrain_loss: 1.1423\n",
      "Iteration: 264 of 277\ttrain_loss: 1.0800\n",
      "Iteration: 266 of 277\ttrain_loss: 1.1899\n",
      "Iteration: 268 of 277\ttrain_loss: 1.0081\n",
      "Iteration: 270 of 277\ttrain_loss: 1.0414\n",
      "Iteration: 272 of 277\ttrain_loss: 1.0892\n",
      "Iteration: 274 of 277\ttrain_loss: 1.1624\n",
      "Iteration: 276 of 277\ttrain_loss: 1.0922\n",
      "Iteration: 277 of 277\ttrain_loss: 1.0546\n",
      "Average Score for this Epoch: 1.090760588645935\n",
      "Eval_loss: 2.5504775047302246\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 31 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 0.9396\n",
      "Iteration: 2 of 277\ttrain_loss: 1.1375\n",
      "Iteration: 4 of 277\ttrain_loss: 0.9764\n",
      "Iteration: 6 of 277\ttrain_loss: 0.9772\n",
      "Iteration: 8 of 277\ttrain_loss: 1.0074\n",
      "Iteration: 10 of 277\ttrain_loss: 0.9616\n",
      "Iteration: 12 of 277\ttrain_loss: 0.9128\n",
      "Iteration: 14 of 277\ttrain_loss: 1.0891\n",
      "Iteration: 16 of 277\ttrain_loss: 1.1319\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1515\n",
      "Iteration: 20 of 277\ttrain_loss: 1.0423\n",
      "Iteration: 22 of 277\ttrain_loss: 1.1175\n",
      "Iteration: 24 of 277\ttrain_loss: 0.9972\n",
      "Iteration: 26 of 277\ttrain_loss: 0.9644\n",
      "Iteration: 28 of 277\ttrain_loss: 1.0179\n",
      "Iteration: 30 of 277\ttrain_loss: 1.0681\n",
      "Iteration: 32 of 277\ttrain_loss: 1.0892\n",
      "Iteration: 34 of 277\ttrain_loss: 0.9849\n",
      "Iteration: 36 of 277\ttrain_loss: 1.0788\n",
      "Iteration: 38 of 277\ttrain_loss: 0.9838\n",
      "Iteration: 40 of 277\ttrain_loss: 1.1408\n",
      "Iteration: 42 of 277\ttrain_loss: 1.0034\n",
      "Iteration: 44 of 277\ttrain_loss: 1.0801\n",
      "Iteration: 46 of 277\ttrain_loss: 1.0687\n",
      "Iteration: 48 of 277\ttrain_loss: 1.0250\n",
      "Iteration: 50 of 277\ttrain_loss: 1.1092\n",
      "Iteration: 52 of 277\ttrain_loss: 1.2061\n",
      "Iteration: 54 of 277\ttrain_loss: 0.9848\n",
      "Iteration: 56 of 277\ttrain_loss: 1.0594\n",
      "Iteration: 58 of 277\ttrain_loss: 1.0825\n",
      "Iteration: 60 of 277\ttrain_loss: 1.1007\n",
      "Iteration: 62 of 277\ttrain_loss: 1.0807\n",
      "Iteration: 64 of 277\ttrain_loss: 1.0541\n",
      "Iteration: 66 of 277\ttrain_loss: 1.0519\n",
      "Iteration: 68 of 277\ttrain_loss: 1.0244\n",
      "Iteration: 70 of 277\ttrain_loss: 1.1377\n",
      "Iteration: 72 of 277\ttrain_loss: 1.0099\n",
      "Iteration: 74 of 277\ttrain_loss: 1.1205\n",
      "Iteration: 76 of 277\ttrain_loss: 1.0289\n",
      "Iteration: 78 of 277\ttrain_loss: 1.0368\n",
      "Iteration: 80 of 277\ttrain_loss: 1.0943\n",
      "Iteration: 82 of 277\ttrain_loss: 1.0664\n",
      "Iteration: 84 of 277\ttrain_loss: 1.0158\n",
      "Iteration: 86 of 277\ttrain_loss: 0.9796\n",
      "Iteration: 88 of 277\ttrain_loss: 1.1744\n",
      "Iteration: 90 of 277\ttrain_loss: 1.1066\n",
      "Iteration: 92 of 277\ttrain_loss: 1.1423\n",
      "Iteration: 94 of 277\ttrain_loss: 2.1301\n",
      "Iteration: 96 of 277\ttrain_loss: 1.1316\n",
      "Iteration: 98 of 277\ttrain_loss: 1.1917\n",
      "Iteration: 100 of 277\ttrain_loss: 1.1636\n",
      "Iteration: 102 of 277\ttrain_loss: 1.1420\n",
      "Iteration: 104 of 277\ttrain_loss: 1.1317\n",
      "Iteration: 106 of 277\ttrain_loss: 1.0957\n",
      "Iteration: 108 of 277\ttrain_loss: 1.1432\n",
      "Iteration: 110 of 277\ttrain_loss: 1.2029\n",
      "Iteration: 112 of 277\ttrain_loss: 1.1480\n",
      "Iteration: 114 of 277\ttrain_loss: 1.2262\n",
      "Iteration: 116 of 277\ttrain_loss: 1.0675\n",
      "Iteration: 118 of 277\ttrain_loss: 1.1554\n",
      "Iteration: 120 of 277\ttrain_loss: 1.0986\n",
      "Iteration: 122 of 277\ttrain_loss: 1.1366\n",
      "Iteration: 124 of 277\ttrain_loss: 1.0213\n",
      "Iteration: 126 of 277\ttrain_loss: 1.1635\n",
      "Iteration: 128 of 277\ttrain_loss: 0.9948\n",
      "Iteration: 130 of 277\ttrain_loss: 1.0378\n",
      "Iteration: 132 of 277\ttrain_loss: 1.0169\n",
      "Iteration: 134 of 277\ttrain_loss: 1.1150\n",
      "Iteration: 136 of 277\ttrain_loss: 1.1916\n",
      "Iteration: 138 of 277\ttrain_loss: 1.1668\n",
      "Iteration: 140 of 277\ttrain_loss: 1.2161\n",
      "Iteration: 142 of 277\ttrain_loss: 1.0038\n",
      "Iteration: 144 of 277\ttrain_loss: 1.1457\n",
      "Iteration: 146 of 277\ttrain_loss: 1.1063\n",
      "Iteration: 148 of 277\ttrain_loss: 1.1946\n",
      "Iteration: 150 of 277\ttrain_loss: 1.0621\n",
      "Iteration: 152 of 277\ttrain_loss: 0.9945\n",
      "Iteration: 154 of 277\ttrain_loss: 1.2587\n",
      "Iteration: 156 of 277\ttrain_loss: 1.0884\n",
      "Iteration: 158 of 277\ttrain_loss: 1.1088\n",
      "Iteration: 160 of 277\ttrain_loss: 1.1018\n",
      "Iteration: 162 of 277\ttrain_loss: 1.3658\n",
      "Iteration: 164 of 277\ttrain_loss: 1.2000\n",
      "Iteration: 166 of 277\ttrain_loss: 1.1988\n",
      "Iteration: 168 of 277\ttrain_loss: 1.0936\n",
      "Iteration: 170 of 277\ttrain_loss: 1.1994\n",
      "Iteration: 172 of 277\ttrain_loss: 1.1682\n",
      "Iteration: 174 of 277\ttrain_loss: 1.2079\n",
      "Iteration: 176 of 277\ttrain_loss: 1.1028\n",
      "Iteration: 178 of 277\ttrain_loss: 1.1119\n",
      "Iteration: 180 of 277\ttrain_loss: 1.1251\n",
      "Iteration: 182 of 277\ttrain_loss: 1.0337\n",
      "Iteration: 184 of 277\ttrain_loss: 1.1051\n",
      "Iteration: 186 of 277\ttrain_loss: 1.1779\n",
      "Iteration: 188 of 277\ttrain_loss: 1.2392\n",
      "Iteration: 190 of 277\ttrain_loss: 1.1430\n",
      "Iteration: 192 of 277\ttrain_loss: 1.2174\n",
      "Iteration: 194 of 277\ttrain_loss: 1.1395\n",
      "Iteration: 196 of 277\ttrain_loss: 1.1625\n",
      "Iteration: 198 of 277\ttrain_loss: 0.9419\n",
      "Iteration: 200 of 277\ttrain_loss: 1.1467\n",
      "Iteration: 202 of 277\ttrain_loss: 1.1647\n",
      "Iteration: 204 of 277\ttrain_loss: 1.0199\n",
      "Iteration: 206 of 277\ttrain_loss: 1.1925\n",
      "Iteration: 208 of 277\ttrain_loss: 1.1135\n",
      "Iteration: 210 of 277\ttrain_loss: 1.1923\n",
      "Iteration: 212 of 277\ttrain_loss: 1.1704\n",
      "Iteration: 214 of 277\ttrain_loss: 1.0779\n",
      "Iteration: 216 of 277\ttrain_loss: 1.2353\n",
      "Iteration: 218 of 277\ttrain_loss: 1.1843\n",
      "Iteration: 220 of 277\ttrain_loss: 1.3113\n",
      "Iteration: 222 of 277\ttrain_loss: 1.2489\n",
      "Iteration: 224 of 277\ttrain_loss: 1.0709\n",
      "Iteration: 226 of 277\ttrain_loss: 1.1707\n",
      "Iteration: 228 of 277\ttrain_loss: 1.1155\n",
      "Iteration: 230 of 277\ttrain_loss: 1.2040\n",
      "Iteration: 232 of 277\ttrain_loss: 1.1460\n",
      "Iteration: 234 of 277\ttrain_loss: 1.2838\n",
      "Iteration: 236 of 277\ttrain_loss: 1.0916\n",
      "Iteration: 238 of 277\ttrain_loss: 1.1812\n",
      "Iteration: 240 of 277\ttrain_loss: 1.2984\n",
      "Iteration: 242 of 277\ttrain_loss: 1.2421\n",
      "Iteration: 244 of 277\ttrain_loss: 1.0742\n",
      "Iteration: 246 of 277\ttrain_loss: 1.1015\n",
      "Iteration: 248 of 277\ttrain_loss: 1.1493\n",
      "Iteration: 250 of 277\ttrain_loss: 1.1720\n",
      "Iteration: 252 of 277\ttrain_loss: 1.1376\n",
      "Iteration: 254 of 277\ttrain_loss: 1.0272\n",
      "Iteration: 256 of 277\ttrain_loss: 1.1170\n",
      "Iteration: 258 of 277\ttrain_loss: 1.0570\n",
      "Iteration: 260 of 277\ttrain_loss: 1.0723\n",
      "Iteration: 262 of 277\ttrain_loss: 0.9691\n",
      "Iteration: 264 of 277\ttrain_loss: 1.4453\n",
      "Iteration: 266 of 277\ttrain_loss: 1.0402\n",
      "Iteration: 268 of 277\ttrain_loss: 1.1975\n",
      "Iteration: 270 of 277\ttrain_loss: 1.2686\n",
      "Iteration: 272 of 277\ttrain_loss: 1.1822\n",
      "Iteration: 274 of 277\ttrain_loss: 1.2176\n",
      "Iteration: 276 of 277\ttrain_loss: 1.2332\n",
      "Iteration: 277 of 277\ttrain_loss: 1.0696\n",
      "Average Score for this Epoch: 1.1187818050384521\n",
      "Eval_loss: 2.5906004905700684\n",
      "\n",
      "-------------------- Epoch 32 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 0.9476\n",
      "Iteration: 2 of 277\ttrain_loss: 1.1776\n",
      "Iteration: 4 of 277\ttrain_loss: 1.1056\n",
      "Iteration: 6 of 277\ttrain_loss: 1.0721\n",
      "Iteration: 8 of 277\ttrain_loss: 1.0955\n",
      "Iteration: 10 of 277\ttrain_loss: 1.1219\n",
      "Iteration: 12 of 277\ttrain_loss: 1.3076\n",
      "Iteration: 14 of 277\ttrain_loss: 1.1070\n",
      "Iteration: 16 of 277\ttrain_loss: 1.1215\n",
      "Iteration: 18 of 277\ttrain_loss: 0.9803\n",
      "Iteration: 20 of 277\ttrain_loss: 1.1756\n",
      "Iteration: 22 of 277\ttrain_loss: 1.1165\n",
      "Iteration: 24 of 277\ttrain_loss: 1.0043\n",
      "Iteration: 26 of 277\ttrain_loss: 0.9683\n",
      "Iteration: 28 of 277\ttrain_loss: 1.1185\n",
      "Iteration: 30 of 277\ttrain_loss: 1.2139\n",
      "Iteration: 32 of 277\ttrain_loss: 1.0611\n",
      "Iteration: 34 of 277\ttrain_loss: 0.9920\n",
      "Iteration: 36 of 277\ttrain_loss: 1.1149\n",
      "Iteration: 38 of 277\ttrain_loss: 1.1543\n",
      "Iteration: 40 of 277\ttrain_loss: 1.0037\n",
      "Iteration: 42 of 277\ttrain_loss: 1.1060\n",
      "Iteration: 44 of 277\ttrain_loss: 1.1834\n",
      "Iteration: 46 of 277\ttrain_loss: 1.1169\n",
      "Iteration: 48 of 277\ttrain_loss: 1.0100\n",
      "Iteration: 50 of 277\ttrain_loss: 1.1328\n",
      "Iteration: 52 of 277\ttrain_loss: 1.0634\n",
      "Iteration: 54 of 277\ttrain_loss: 1.0766\n",
      "Iteration: 56 of 277\ttrain_loss: 1.1687\n",
      "Iteration: 58 of 277\ttrain_loss: 1.0872\n",
      "Iteration: 60 of 277\ttrain_loss: 0.9908\n",
      "Iteration: 62 of 277\ttrain_loss: 1.1480\n",
      "Iteration: 64 of 277\ttrain_loss: 1.0497\n",
      "Iteration: 66 of 277\ttrain_loss: 1.0706\n",
      "Iteration: 68 of 277\ttrain_loss: 0.9566\n",
      "Iteration: 70 of 277\ttrain_loss: 1.1017\n",
      "Iteration: 72 of 277\ttrain_loss: 1.1757\n",
      "Iteration: 74 of 277\ttrain_loss: 1.1136\n",
      "Iteration: 76 of 277\ttrain_loss: 1.0477\n",
      "Iteration: 78 of 277\ttrain_loss: 0.9944\n",
      "Iteration: 80 of 277\ttrain_loss: 1.1103\n",
      "Iteration: 82 of 277\ttrain_loss: 1.2817\n",
      "Iteration: 84 of 277\ttrain_loss: 1.0561\n",
      "Iteration: 86 of 277\ttrain_loss: 1.0887\n",
      "Iteration: 88 of 277\ttrain_loss: 1.1768\n",
      "Iteration: 90 of 277\ttrain_loss: 1.1040\n",
      "Iteration: 92 of 277\ttrain_loss: 1.2328\n",
      "Iteration: 94 of 277\ttrain_loss: 1.3273\n",
      "Iteration: 96 of 277\ttrain_loss: 1.1188\n",
      "Iteration: 98 of 277\ttrain_loss: 1.1013\n",
      "Iteration: 100 of 277\ttrain_loss: 1.1174\n",
      "Iteration: 102 of 277\ttrain_loss: 1.0496\n",
      "Iteration: 104 of 277\ttrain_loss: 1.1843\n",
      "Iteration: 106 of 277\ttrain_loss: 1.0793\n",
      "Iteration: 108 of 277\ttrain_loss: 1.0136\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0598\n",
      "Iteration: 112 of 277\ttrain_loss: 1.1703\n",
      "Iteration: 114 of 277\ttrain_loss: 1.1028\n",
      "Iteration: 116 of 277\ttrain_loss: 1.2289\n",
      "Iteration: 118 of 277\ttrain_loss: 1.1852\n",
      "Iteration: 120 of 277\ttrain_loss: 1.1799\n",
      "Iteration: 122 of 277\ttrain_loss: 1.0617\n",
      "Iteration: 124 of 277\ttrain_loss: 1.1641\n",
      "Iteration: 126 of 277\ttrain_loss: 1.1645\n",
      "Iteration: 128 of 277\ttrain_loss: 1.1173\n",
      "Iteration: 130 of 277\ttrain_loss: 1.0748\n",
      "Iteration: 132 of 277\ttrain_loss: 1.1414\n",
      "Iteration: 134 of 277\ttrain_loss: 1.1296\n",
      "Iteration: 136 of 277\ttrain_loss: 1.1329\n",
      "Iteration: 138 of 277\ttrain_loss: 1.2171\n",
      "Iteration: 140 of 277\ttrain_loss: 1.1060\n",
      "Iteration: 142 of 277\ttrain_loss: 1.1176\n",
      "Iteration: 144 of 277\ttrain_loss: 1.2133\n",
      "Iteration: 146 of 277\ttrain_loss: 1.1587\n",
      "Iteration: 148 of 277\ttrain_loss: 1.1033\n",
      "Iteration: 150 of 277\ttrain_loss: 1.1414\n",
      "Iteration: 152 of 277\ttrain_loss: 1.1944\n",
      "Iteration: 154 of 277\ttrain_loss: 1.1943\n",
      "Iteration: 156 of 277\ttrain_loss: 1.0686\n",
      "Iteration: 158 of 277\ttrain_loss: 1.1947\n",
      "Iteration: 160 of 277\ttrain_loss: 1.2210\n",
      "Iteration: 162 of 277\ttrain_loss: 1.2600\n",
      "Iteration: 164 of 277\ttrain_loss: 1.2534\n",
      "Iteration: 166 of 277\ttrain_loss: 1.2691\n",
      "Iteration: 168 of 277\ttrain_loss: 1.0920\n",
      "Iteration: 170 of 277\ttrain_loss: 1.3413\n",
      "Iteration: 172 of 277\ttrain_loss: 1.1316\n",
      "Iteration: 174 of 277\ttrain_loss: 1.0748\n",
      "Iteration: 176 of 277\ttrain_loss: 1.2048\n",
      "Iteration: 178 of 277\ttrain_loss: 1.2328\n",
      "Iteration: 180 of 277\ttrain_loss: 1.1201\n",
      "Iteration: 182 of 277\ttrain_loss: 1.1062\n",
      "Iteration: 184 of 277\ttrain_loss: 1.2014\n",
      "Iteration: 186 of 277\ttrain_loss: 1.3481\n",
      "Iteration: 188 of 277\ttrain_loss: 1.1446\n",
      "Iteration: 190 of 277\ttrain_loss: 1.3323\n",
      "Iteration: 192 of 277\ttrain_loss: 1.1453\n",
      "Iteration: 194 of 277\ttrain_loss: 1.2179\n",
      "Iteration: 196 of 277\ttrain_loss: 1.2124\n",
      "Iteration: 198 of 277\ttrain_loss: 1.0936\n",
      "Iteration: 200 of 277\ttrain_loss: 1.1340\n",
      "Iteration: 202 of 277\ttrain_loss: 1.4551\n",
      "Iteration: 204 of 277\ttrain_loss: 1.2423\n",
      "Iteration: 206 of 277\ttrain_loss: 1.0912\n",
      "Iteration: 208 of 277\ttrain_loss: 1.2262\n",
      "Iteration: 210 of 277\ttrain_loss: 1.3165\n",
      "Iteration: 212 of 277\ttrain_loss: 2.2241\n",
      "Iteration: 214 of 277\ttrain_loss: 1.2971\n",
      "Iteration: 216 of 277\ttrain_loss: 1.1762\n",
      "Iteration: 218 of 277\ttrain_loss: 1.2535\n",
      "Iteration: 220 of 277\ttrain_loss: 1.5060\n",
      "Iteration: 222 of 277\ttrain_loss: 1.1456\n",
      "Iteration: 224 of 277\ttrain_loss: 1.3348\n",
      "Iteration: 226 of 277\ttrain_loss: 1.2579\n",
      "Iteration: 228 of 277\ttrain_loss: 1.1617\n",
      "Iteration: 230 of 277\ttrain_loss: 1.2412\n",
      "Iteration: 232 of 277\ttrain_loss: 1.4079\n",
      "Iteration: 234 of 277\ttrain_loss: 1.1715\n",
      "Iteration: 236 of 277\ttrain_loss: 1.3074\n",
      "Iteration: 238 of 277\ttrain_loss: 1.1572\n",
      "Iteration: 240 of 277\ttrain_loss: 1.4255\n",
      "Iteration: 242 of 277\ttrain_loss: 1.4010\n",
      "Iteration: 244 of 277\ttrain_loss: 1.4173\n",
      "Iteration: 246 of 277\ttrain_loss: 1.3360\n",
      "Iteration: 248 of 277\ttrain_loss: 1.2375\n",
      "Iteration: 250 of 277\ttrain_loss: 1.3064\n",
      "Iteration: 252 of 277\ttrain_loss: 1.1939\n",
      "Iteration: 254 of 277\ttrain_loss: 1.3253\n",
      "Iteration: 256 of 277\ttrain_loss: 1.2933\n",
      "Iteration: 258 of 277\ttrain_loss: 1.1755\n",
      "Iteration: 260 of 277\ttrain_loss: 1.2647\n",
      "Iteration: 262 of 277\ttrain_loss: 1.3295\n",
      "Iteration: 264 of 277\ttrain_loss: 1.2728\n",
      "Iteration: 266 of 277\ttrain_loss: 1.3659\n",
      "Iteration: 268 of 277\ttrain_loss: 1.1266\n",
      "Iteration: 270 of 277\ttrain_loss: 1.2999\n",
      "Iteration: 272 of 277\ttrain_loss: 1.2264\n",
      "Iteration: 274 of 277\ttrain_loss: 1.2935\n",
      "Iteration: 276 of 277\ttrain_loss: 1.0655\n",
      "Iteration: 277 of 277\ttrain_loss: 0.9996\n",
      "Average Score for this Epoch: 1.1844232082366943\n",
      "Eval_loss: 2.622607707977295\n",
      "\n",
      "-------------------- Epoch 33 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.2047\n",
      "Iteration: 2 of 277\ttrain_loss: 1.0926\n",
      "Iteration: 4 of 277\ttrain_loss: 1.3469\n",
      "Iteration: 6 of 277\ttrain_loss: 1.0972\n",
      "Iteration: 8 of 277\ttrain_loss: 1.1777\n",
      "Iteration: 10 of 277\ttrain_loss: 1.1315\n",
      "Iteration: 12 of 277\ttrain_loss: 1.1279\n",
      "Iteration: 14 of 277\ttrain_loss: 1.2827\n",
      "Iteration: 16 of 277\ttrain_loss: 0.9576\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1411\n",
      "Iteration: 20 of 277\ttrain_loss: 1.1698\n",
      "Iteration: 22 of 277\ttrain_loss: 1.0415\n",
      "Iteration: 24 of 277\ttrain_loss: 1.0642\n",
      "Iteration: 26 of 277\ttrain_loss: 1.1863\n",
      "Iteration: 28 of 277\ttrain_loss: 1.2017\n",
      "Iteration: 30 of 277\ttrain_loss: 1.1647\n",
      "Iteration: 32 of 277\ttrain_loss: 1.1120\n",
      "Iteration: 34 of 277\ttrain_loss: 1.2480\n",
      "Iteration: 36 of 277\ttrain_loss: 1.1218\n",
      "Iteration: 38 of 277\ttrain_loss: 1.1451\n",
      "Iteration: 40 of 277\ttrain_loss: 1.1851\n",
      "Iteration: 42 of 277\ttrain_loss: 1.1476\n",
      "Iteration: 44 of 277\ttrain_loss: 0.9783\n",
      "Iteration: 46 of 277\ttrain_loss: 1.2792\n",
      "Iteration: 48 of 277\ttrain_loss: 1.1875\n",
      "Iteration: 50 of 277\ttrain_loss: 1.2044\n",
      "Iteration: 52 of 277\ttrain_loss: 1.2654\n",
      "Iteration: 54 of 277\ttrain_loss: 1.1603\n",
      "Iteration: 56 of 277\ttrain_loss: 1.3265\n",
      "Iteration: 58 of 277\ttrain_loss: 1.1248\n",
      "Iteration: 60 of 277\ttrain_loss: 1.1243\n",
      "Iteration: 62 of 277\ttrain_loss: 1.1403\n",
      "Iteration: 64 of 277\ttrain_loss: 1.1098\n",
      "Iteration: 66 of 277\ttrain_loss: 1.4523\n",
      "Iteration: 68 of 277\ttrain_loss: 1.2284\n",
      "Iteration: 70 of 277\ttrain_loss: 1.1540\n",
      "Iteration: 72 of 277\ttrain_loss: 1.0538\n",
      "Iteration: 74 of 277\ttrain_loss: 1.1458\n",
      "Iteration: 76 of 277\ttrain_loss: 1.1487\n",
      "Iteration: 78 of 277\ttrain_loss: 1.1339\n",
      "Iteration: 80 of 277\ttrain_loss: 1.2373\n",
      "Iteration: 82 of 277\ttrain_loss: 1.1416\n",
      "Iteration: 84 of 277\ttrain_loss: 1.1986\n",
      "Iteration: 86 of 277\ttrain_loss: 1.0785\n",
      "Iteration: 88 of 277\ttrain_loss: 0.9411\n",
      "Iteration: 90 of 277\ttrain_loss: 1.0416\n",
      "Iteration: 92 of 277\ttrain_loss: 1.0587\n",
      "Iteration: 94 of 277\ttrain_loss: 1.1690\n",
      "Iteration: 96 of 277\ttrain_loss: 0.9892\n",
      "Iteration: 98 of 277\ttrain_loss: 1.0342\n",
      "Iteration: 100 of 277\ttrain_loss: 1.2655\n",
      "Iteration: 102 of 277\ttrain_loss: 1.3399\n",
      "Iteration: 104 of 277\ttrain_loss: 1.2094\n",
      "Iteration: 106 of 277\ttrain_loss: 1.1913\n",
      "Iteration: 108 of 277\ttrain_loss: 1.2748\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0802\n",
      "Iteration: 112 of 277\ttrain_loss: 1.1453\n",
      "Iteration: 114 of 277\ttrain_loss: 1.1323\n",
      "Iteration: 116 of 277\ttrain_loss: 1.2366\n",
      "Iteration: 118 of 277\ttrain_loss: 1.1373\n",
      "Iteration: 120 of 277\ttrain_loss: 1.0292\n",
      "Iteration: 122 of 277\ttrain_loss: 1.3096\n",
      "Iteration: 124 of 277\ttrain_loss: 1.3479\n",
      "Iteration: 126 of 277\ttrain_loss: 2.2080\n",
      "Iteration: 128 of 277\ttrain_loss: 1.2335\n",
      "Iteration: 130 of 277\ttrain_loss: 1.0666\n",
      "Iteration: 132 of 277\ttrain_loss: 1.0424\n",
      "Iteration: 134 of 277\ttrain_loss: 1.0421\n",
      "Iteration: 136 of 277\ttrain_loss: 1.1646\n",
      "Iteration: 138 of 277\ttrain_loss: 1.2345\n",
      "Iteration: 140 of 277\ttrain_loss: 1.2088\n",
      "Iteration: 142 of 277\ttrain_loss: 1.0962\n",
      "Iteration: 144 of 277\ttrain_loss: 1.1762\n",
      "Iteration: 146 of 277\ttrain_loss: 1.2067\n",
      "Iteration: 148 of 277\ttrain_loss: 1.0679\n",
      "Iteration: 150 of 277\ttrain_loss: 1.2407\n",
      "Iteration: 152 of 277\ttrain_loss: 1.1439\n",
      "Iteration: 154 of 277\ttrain_loss: 1.1508\n",
      "Iteration: 156 of 277\ttrain_loss: 1.1140\n",
      "Iteration: 158 of 277\ttrain_loss: 1.3761\n",
      "Iteration: 160 of 277\ttrain_loss: 1.1011\n",
      "Iteration: 162 of 277\ttrain_loss: 1.2222\n",
      "Iteration: 164 of 277\ttrain_loss: 1.0960\n",
      "Iteration: 166 of 277\ttrain_loss: 1.1440\n",
      "Iteration: 168 of 277\ttrain_loss: 1.1876\n",
      "Iteration: 170 of 277\ttrain_loss: 1.1097\n",
      "Iteration: 172 of 277\ttrain_loss: 1.1033\n",
      "Iteration: 174 of 277\ttrain_loss: 1.4284\n",
      "Iteration: 176 of 277\ttrain_loss: 1.1796\n",
      "Iteration: 178 of 277\ttrain_loss: 1.1641\n",
      "Iteration: 180 of 277\ttrain_loss: 1.3129\n",
      "Iteration: 182 of 277\ttrain_loss: 1.1119\n",
      "Iteration: 184 of 277\ttrain_loss: 1.1841\n",
      "Iteration: 186 of 277\ttrain_loss: 1.0827\n",
      "Iteration: 188 of 277\ttrain_loss: 1.0856\n",
      "Iteration: 190 of 277\ttrain_loss: 1.1156\n",
      "Iteration: 192 of 277\ttrain_loss: 1.0111\n",
      "Iteration: 194 of 277\ttrain_loss: 1.2251\n",
      "Iteration: 196 of 277\ttrain_loss: 1.3006\n",
      "Iteration: 198 of 277\ttrain_loss: 1.0822\n",
      "Iteration: 200 of 277\ttrain_loss: 1.0761\n",
      "Iteration: 202 of 277\ttrain_loss: 1.1802\n",
      "Iteration: 204 of 277\ttrain_loss: 1.0803\n",
      "Iteration: 206 of 277\ttrain_loss: 1.0659\n",
      "Iteration: 208 of 277\ttrain_loss: 1.1153\n",
      "Iteration: 210 of 277\ttrain_loss: 1.1043\n",
      "Iteration: 212 of 277\ttrain_loss: 1.1265\n",
      "Iteration: 214 of 277\ttrain_loss: 1.0877\n",
      "Iteration: 216 of 277\ttrain_loss: 1.0584\n",
      "Iteration: 218 of 277\ttrain_loss: 1.1386\n",
      "Iteration: 220 of 277\ttrain_loss: 1.2162\n",
      "Iteration: 222 of 277\ttrain_loss: 1.1080\n",
      "Iteration: 224 of 277\ttrain_loss: 1.0856\n",
      "Iteration: 226 of 277\ttrain_loss: 1.3135\n",
      "Iteration: 228 of 277\ttrain_loss: 1.3141\n",
      "Iteration: 230 of 277\ttrain_loss: 1.1322\n",
      "Iteration: 232 of 277\ttrain_loss: 1.0845\n",
      "Iteration: 234 of 277\ttrain_loss: 1.2651\n",
      "Iteration: 236 of 277\ttrain_loss: 1.2358\n",
      "Iteration: 238 of 277\ttrain_loss: 1.1131\n",
      "Iteration: 240 of 277\ttrain_loss: 1.2299\n",
      "Iteration: 242 of 277\ttrain_loss: 1.0681\n",
      "Iteration: 244 of 277\ttrain_loss: 1.2354\n",
      "Iteration: 246 of 277\ttrain_loss: 1.1815\n",
      "Iteration: 248 of 277\ttrain_loss: 1.3861\n",
      "Iteration: 250 of 277\ttrain_loss: 1.2075\n",
      "Iteration: 252 of 277\ttrain_loss: 1.1924\n",
      "Iteration: 254 of 277\ttrain_loss: 1.1899\n",
      "Iteration: 256 of 277\ttrain_loss: 1.0544\n",
      "Iteration: 258 of 277\ttrain_loss: 1.1125\n",
      "Iteration: 260 of 277\ttrain_loss: 1.2553\n",
      "Iteration: 262 of 277\ttrain_loss: 1.2106\n",
      "Iteration: 264 of 277\ttrain_loss: 0.9875\n",
      "Iteration: 266 of 277\ttrain_loss: 1.1615\n",
      "Iteration: 268 of 277\ttrain_loss: 1.0368\n",
      "Iteration: 270 of 277\ttrain_loss: 0.9959\n",
      "Iteration: 272 of 277\ttrain_loss: 1.1399\n",
      "Iteration: 274 of 277\ttrain_loss: 1.2499\n",
      "Iteration: 276 of 277\ttrain_loss: 1.3118\n",
      "Iteration: 277 of 277\ttrain_loss: 1.1044\n",
      "Average Score for this Epoch: 1.1662031412124634\n",
      "Eval_loss: 2.6078014373779297\n",
      "\n",
      "-------------------- Epoch 34 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.0334\n",
      "Iteration: 2 of 277\ttrain_loss: 1.1073\n",
      "Iteration: 4 of 277\ttrain_loss: 0.9824\n",
      "Iteration: 6 of 277\ttrain_loss: 1.0381\n",
      "Iteration: 8 of 277\ttrain_loss: 0.9585\n",
      "Iteration: 10 of 277\ttrain_loss: 1.0044\n",
      "Iteration: 12 of 277\ttrain_loss: 1.0534\n",
      "Iteration: 14 of 277\ttrain_loss: 1.1322\n",
      "Iteration: 16 of 277\ttrain_loss: 0.9945\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1894\n",
      "Iteration: 20 of 277\ttrain_loss: 1.0698\n",
      "Iteration: 22 of 277\ttrain_loss: 1.1747\n",
      "Iteration: 24 of 277\ttrain_loss: 1.2593\n",
      "Iteration: 26 of 277\ttrain_loss: 1.2027\n",
      "Iteration: 28 of 277\ttrain_loss: 1.0937\n",
      "Iteration: 30 of 277\ttrain_loss: 1.0245\n",
      "Iteration: 32 of 277\ttrain_loss: 1.0609\n",
      "Iteration: 34 of 277\ttrain_loss: 0.9020\n",
      "Iteration: 36 of 277\ttrain_loss: 0.9599\n",
      "Iteration: 38 of 277\ttrain_loss: 1.1075\n",
      "Iteration: 40 of 277\ttrain_loss: 0.9973\n",
      "Iteration: 42 of 277\ttrain_loss: 0.9738\n",
      "Iteration: 44 of 277\ttrain_loss: 1.1432\n",
      "Iteration: 46 of 277\ttrain_loss: 1.2202\n",
      "Iteration: 48 of 277\ttrain_loss: 0.9827\n",
      "Iteration: 50 of 277\ttrain_loss: 1.0447\n",
      "Iteration: 52 of 277\ttrain_loss: 1.0221\n",
      "Iteration: 54 of 277\ttrain_loss: 0.9497\n",
      "Iteration: 56 of 277\ttrain_loss: 1.0293\n",
      "Iteration: 58 of 277\ttrain_loss: 1.0269\n",
      "Iteration: 60 of 277\ttrain_loss: 1.0919\n",
      "Iteration: 62 of 277\ttrain_loss: 1.0532\n",
      "Iteration: 64 of 277\ttrain_loss: 1.1146\n",
      "Iteration: 66 of 277\ttrain_loss: 1.1761\n",
      "Iteration: 68 of 277\ttrain_loss: 1.0957\n",
      "Iteration: 70 of 277\ttrain_loss: 1.2313\n",
      "Iteration: 72 of 277\ttrain_loss: 1.0364\n",
      "Iteration: 74 of 277\ttrain_loss: 1.1711\n",
      "Iteration: 76 of 277\ttrain_loss: 1.0533\n",
      "Iteration: 78 of 277\ttrain_loss: 1.0536\n",
      "Iteration: 80 of 277\ttrain_loss: 0.9146\n",
      "Iteration: 82 of 277\ttrain_loss: 1.1106\n",
      "Iteration: 84 of 277\ttrain_loss: 1.1115\n",
      "Iteration: 86 of 277\ttrain_loss: 1.1686\n",
      "Iteration: 88 of 277\ttrain_loss: 0.9332\n",
      "Iteration: 90 of 277\ttrain_loss: 1.1535\n",
      "Iteration: 92 of 277\ttrain_loss: 0.9733\n",
      "Iteration: 94 of 277\ttrain_loss: 1.1741\n",
      "Iteration: 96 of 277\ttrain_loss: 0.9199\n",
      "Iteration: 98 of 277\ttrain_loss: 1.0432\n",
      "Iteration: 100 of 277\ttrain_loss: 1.1770\n",
      "Iteration: 102 of 277\ttrain_loss: 1.0267\n",
      "Iteration: 104 of 277\ttrain_loss: 0.9528\n",
      "Iteration: 106 of 277\ttrain_loss: 1.0108\n",
      "Iteration: 108 of 277\ttrain_loss: 0.9108\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0217\n",
      "Iteration: 112 of 277\ttrain_loss: 1.0006\n",
      "Iteration: 114 of 277\ttrain_loss: 1.2204\n",
      "Iteration: 116 of 277\ttrain_loss: 1.0617\n",
      "Iteration: 118 of 277\ttrain_loss: 1.0023\n",
      "Iteration: 120 of 277\ttrain_loss: 1.0782\n",
      "Iteration: 122 of 277\ttrain_loss: 1.0748\n",
      "Iteration: 124 of 277\ttrain_loss: 0.9157\n",
      "Iteration: 126 of 277\ttrain_loss: 1.0786\n",
      "Iteration: 128 of 277\ttrain_loss: 1.2056\n",
      "Iteration: 130 of 277\ttrain_loss: 1.0188\n",
      "Iteration: 132 of 277\ttrain_loss: 1.0372\n",
      "Iteration: 134 of 277\ttrain_loss: 1.1979\n",
      "Iteration: 136 of 277\ttrain_loss: 1.1449\n",
      "Iteration: 138 of 277\ttrain_loss: 1.1293\n",
      "Iteration: 140 of 277\ttrain_loss: 1.0343\n",
      "Iteration: 142 of 277\ttrain_loss: 1.1120\n",
      "Iteration: 144 of 277\ttrain_loss: 1.2543\n",
      "Iteration: 146 of 277\ttrain_loss: 1.0250\n",
      "Iteration: 148 of 277\ttrain_loss: 0.9695\n",
      "Iteration: 150 of 277\ttrain_loss: 1.0811\n",
      "Iteration: 152 of 277\ttrain_loss: 0.9717\n",
      "Iteration: 154 of 277\ttrain_loss: 1.0045\n",
      "Iteration: 156 of 277\ttrain_loss: 1.1135\n",
      "Iteration: 158 of 277\ttrain_loss: 1.0972\n",
      "Iteration: 160 of 277\ttrain_loss: 1.0484\n",
      "Iteration: 162 of 277\ttrain_loss: 1.0690\n",
      "Iteration: 164 of 277\ttrain_loss: 1.0070\n",
      "Iteration: 166 of 277\ttrain_loss: 1.0941\n",
      "Iteration: 168 of 277\ttrain_loss: 1.0393\n",
      "Iteration: 170 of 277\ttrain_loss: 1.0726\n",
      "Iteration: 172 of 277\ttrain_loss: 0.9497\n",
      "Iteration: 174 of 277\ttrain_loss: 1.0546\n",
      "Iteration: 176 of 277\ttrain_loss: 1.0651\n",
      "Iteration: 178 of 277\ttrain_loss: 0.9124\n",
      "Iteration: 180 of 277\ttrain_loss: 1.0292\n",
      "Iteration: 182 of 277\ttrain_loss: 1.0243\n",
      "Iteration: 184 of 277\ttrain_loss: 1.1980\n",
      "Iteration: 186 of 277\ttrain_loss: 1.0468\n",
      "Iteration: 188 of 277\ttrain_loss: 0.9869\n",
      "Iteration: 190 of 277\ttrain_loss: 0.9657\n",
      "Iteration: 192 of 277\ttrain_loss: 1.1106\n",
      "Iteration: 194 of 277\ttrain_loss: 1.1454\n",
      "Iteration: 196 of 277\ttrain_loss: 1.0088\n",
      "Iteration: 198 of 277\ttrain_loss: 0.9629\n",
      "Iteration: 200 of 277\ttrain_loss: 1.1820\n",
      "Iteration: 202 of 277\ttrain_loss: 1.0839\n",
      "Iteration: 204 of 277\ttrain_loss: 1.0102\n",
      "Iteration: 206 of 277\ttrain_loss: 1.1244\n",
      "Iteration: 208 of 277\ttrain_loss: 1.0763\n",
      "Iteration: 210 of 277\ttrain_loss: 1.1522\n",
      "Iteration: 212 of 277\ttrain_loss: 1.2994\n",
      "Iteration: 214 of 277\ttrain_loss: 1.0124\n",
      "Iteration: 216 of 277\ttrain_loss: 1.0097\n",
      "Iteration: 218 of 277\ttrain_loss: 1.0333\n",
      "Iteration: 220 of 277\ttrain_loss: 0.9457\n",
      "Iteration: 222 of 277\ttrain_loss: 1.1540\n",
      "Iteration: 224 of 277\ttrain_loss: 1.0073\n",
      "Iteration: 226 of 277\ttrain_loss: 1.0244\n",
      "Iteration: 228 of 277\ttrain_loss: 1.0126\n",
      "Iteration: 230 of 277\ttrain_loss: 1.0859\n",
      "Iteration: 232 of 277\ttrain_loss: 1.0149\n",
      "Iteration: 234 of 277\ttrain_loss: 1.0045\n",
      "Iteration: 236 of 277\ttrain_loss: 1.3426\n",
      "Iteration: 238 of 277\ttrain_loss: 1.0540\n",
      "Iteration: 240 of 277\ttrain_loss: 0.9129\n",
      "Iteration: 242 of 277\ttrain_loss: 1.0139\n",
      "Iteration: 244 of 277\ttrain_loss: 1.1439\n",
      "Iteration: 246 of 277\ttrain_loss: 1.4276\n",
      "Iteration: 248 of 277\ttrain_loss: 1.1408\n",
      "Iteration: 250 of 277\ttrain_loss: 1.0168\n",
      "Iteration: 252 of 277\ttrain_loss: 1.0953\n",
      "Iteration: 254 of 277\ttrain_loss: 1.0456\n",
      "Iteration: 256 of 277\ttrain_loss: 0.9736\n",
      "Iteration: 258 of 277\ttrain_loss: 1.2222\n",
      "Iteration: 260 of 277\ttrain_loss: 1.0266\n",
      "Iteration: 262 of 277\ttrain_loss: 0.9821\n",
      "Iteration: 264 of 277\ttrain_loss: 1.0819\n",
      "Iteration: 266 of 277\ttrain_loss: 1.0838\n",
      "Iteration: 268 of 277\ttrain_loss: 0.9701\n",
      "Iteration: 270 of 277\ttrain_loss: 1.1971\n",
      "Iteration: 272 of 277\ttrain_loss: 1.0756\n",
      "Iteration: 274 of 277\ttrain_loss: 1.0717\n",
      "Iteration: 276 of 277\ttrain_loss: 1.0125\n",
      "Iteration: 277 of 277\ttrain_loss: 0.9921\n",
      "Average Score for this Epoch: 1.067819595336914\n",
      "Eval_loss: 2.6084563732147217\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 35 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.0545\n",
      "Iteration: 2 of 277\ttrain_loss: 1.0697\n",
      "Iteration: 4 of 277\ttrain_loss: 0.9672\n",
      "Iteration: 6 of 277\ttrain_loss: 1.0621\n",
      "Iteration: 8 of 277\ttrain_loss: 0.8408\n",
      "Iteration: 10 of 277\ttrain_loss: 0.9805\n",
      "Iteration: 12 of 277\ttrain_loss: 0.9752\n",
      "Iteration: 14 of 277\ttrain_loss: 0.9702\n",
      "Iteration: 16 of 277\ttrain_loss: 0.9399\n",
      "Iteration: 18 of 277\ttrain_loss: 0.9843\n",
      "Iteration: 20 of 277\ttrain_loss: 0.9599\n",
      "Iteration: 22 of 277\ttrain_loss: 0.8965\n",
      "Iteration: 24 of 277\ttrain_loss: 0.9381\n",
      "Iteration: 26 of 277\ttrain_loss: 0.9529\n",
      "Iteration: 28 of 277\ttrain_loss: 0.8804\n",
      "Iteration: 30 of 277\ttrain_loss: 1.0006\n",
      "Iteration: 32 of 277\ttrain_loss: 0.9839\n",
      "Iteration: 34 of 277\ttrain_loss: 0.9584\n",
      "Iteration: 36 of 277\ttrain_loss: 0.9798\n",
      "Iteration: 38 of 277\ttrain_loss: 0.9786\n",
      "Iteration: 40 of 277\ttrain_loss: 1.1216\n",
      "Iteration: 42 of 277\ttrain_loss: 0.9810\n",
      "Iteration: 44 of 277\ttrain_loss: 0.9181\n",
      "Iteration: 46 of 277\ttrain_loss: 1.3318\n",
      "Iteration: 48 of 277\ttrain_loss: 0.9854\n",
      "Iteration: 50 of 277\ttrain_loss: 1.0832\n",
      "Iteration: 52 of 277\ttrain_loss: 0.9364\n",
      "Iteration: 54 of 277\ttrain_loss: 0.9826\n",
      "Iteration: 56 of 277\ttrain_loss: 0.9448\n",
      "Iteration: 58 of 277\ttrain_loss: 1.0137\n",
      "Iteration: 60 of 277\ttrain_loss: 1.0295\n",
      "Iteration: 62 of 277\ttrain_loss: 0.8609\n",
      "Iteration: 64 of 277\ttrain_loss: 0.9160\n",
      "Iteration: 66 of 277\ttrain_loss: 1.1004\n",
      "Iteration: 68 of 277\ttrain_loss: 1.0475\n",
      "Iteration: 70 of 277\ttrain_loss: 0.9419\n",
      "Iteration: 72 of 277\ttrain_loss: 0.9517\n",
      "Iteration: 74 of 277\ttrain_loss: 1.0373\n",
      "Iteration: 76 of 277\ttrain_loss: 0.9910\n",
      "Iteration: 78 of 277\ttrain_loss: 0.9906\n",
      "Iteration: 80 of 277\ttrain_loss: 0.9886\n",
      "Iteration: 82 of 277\ttrain_loss: 0.8663\n",
      "Iteration: 84 of 277\ttrain_loss: 0.9099\n",
      "Iteration: 86 of 277\ttrain_loss: 0.9717\n",
      "Iteration: 88 of 277\ttrain_loss: 1.0324\n",
      "Iteration: 90 of 277\ttrain_loss: 0.9433\n",
      "Iteration: 92 of 277\ttrain_loss: 1.0601\n",
      "Iteration: 94 of 277\ttrain_loss: 1.0467\n",
      "Iteration: 96 of 277\ttrain_loss: 0.9469\n",
      "Iteration: 98 of 277\ttrain_loss: 0.9263\n",
      "Iteration: 100 of 277\ttrain_loss: 1.0272\n",
      "Iteration: 102 of 277\ttrain_loss: 1.0168\n",
      "Iteration: 104 of 277\ttrain_loss: 1.1303\n",
      "Iteration: 106 of 277\ttrain_loss: 1.0824\n",
      "Iteration: 108 of 277\ttrain_loss: 1.0651\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0267\n",
      "Iteration: 112 of 277\ttrain_loss: 1.0590\n",
      "Iteration: 114 of 277\ttrain_loss: 0.9384\n",
      "Iteration: 116 of 277\ttrain_loss: 0.9805\n",
      "Iteration: 118 of 277\ttrain_loss: 1.0685\n",
      "Iteration: 120 of 277\ttrain_loss: 1.0467\n",
      "Iteration: 122 of 277\ttrain_loss: 2.0819\n",
      "Iteration: 124 of 277\ttrain_loss: 1.0134\n",
      "Iteration: 126 of 277\ttrain_loss: 1.0823\n",
      "Iteration: 128 of 277\ttrain_loss: 0.9800\n",
      "Iteration: 130 of 277\ttrain_loss: 1.0909\n",
      "Iteration: 132 of 277\ttrain_loss: 0.9513\n",
      "Iteration: 134 of 277\ttrain_loss: 0.8943\n",
      "Iteration: 136 of 277\ttrain_loss: 0.9923\n",
      "Iteration: 138 of 277\ttrain_loss: 1.0647\n",
      "Iteration: 140 of 277\ttrain_loss: 0.9518\n",
      "Iteration: 142 of 277\ttrain_loss: 1.0226\n",
      "Iteration: 144 of 277\ttrain_loss: 0.9841\n",
      "Iteration: 146 of 277\ttrain_loss: 1.0123\n",
      "Iteration: 148 of 277\ttrain_loss: 0.9551\n",
      "Iteration: 150 of 277\ttrain_loss: 1.1156\n",
      "Iteration: 152 of 277\ttrain_loss: 1.0198\n",
      "Iteration: 154 of 277\ttrain_loss: 0.9633\n",
      "Iteration: 156 of 277\ttrain_loss: 0.9567\n",
      "Iteration: 158 of 277\ttrain_loss: 1.1934\n",
      "Iteration: 160 of 277\ttrain_loss: 0.9421\n",
      "Iteration: 162 of 277\ttrain_loss: 0.9241\n",
      "Iteration: 164 of 277\ttrain_loss: 0.9661\n",
      "Iteration: 166 of 277\ttrain_loss: 0.9795\n",
      "Iteration: 168 of 277\ttrain_loss: 1.0351\n",
      "Iteration: 170 of 277\ttrain_loss: 0.9944\n",
      "Iteration: 172 of 277\ttrain_loss: 0.9823\n",
      "Iteration: 174 of 277\ttrain_loss: 0.9689\n",
      "Iteration: 176 of 277\ttrain_loss: 0.9705\n",
      "Iteration: 178 of 277\ttrain_loss: 1.0067\n",
      "Iteration: 180 of 277\ttrain_loss: 0.9402\n",
      "Iteration: 182 of 277\ttrain_loss: 1.0084\n",
      "Iteration: 184 of 277\ttrain_loss: 0.9786\n",
      "Iteration: 186 of 277\ttrain_loss: 1.1147\n",
      "Iteration: 188 of 277\ttrain_loss: 0.9115\n",
      "Iteration: 190 of 277\ttrain_loss: 1.0227\n",
      "Iteration: 192 of 277\ttrain_loss: 1.1683\n",
      "Iteration: 194 of 277\ttrain_loss: 0.9509\n",
      "Iteration: 196 of 277\ttrain_loss: 1.0259\n",
      "Iteration: 198 of 277\ttrain_loss: 1.1345\n",
      "Iteration: 200 of 277\ttrain_loss: 1.0428\n",
      "Iteration: 202 of 277\ttrain_loss: 0.9013\n",
      "Iteration: 204 of 277\ttrain_loss: 0.9700\n",
      "Iteration: 206 of 277\ttrain_loss: 0.9835\n",
      "Iteration: 208 of 277\ttrain_loss: 0.8835\n",
      "Iteration: 210 of 277\ttrain_loss: 0.9986\n",
      "Iteration: 212 of 277\ttrain_loss: 1.0043\n",
      "Iteration: 214 of 277\ttrain_loss: 1.0336\n",
      "Iteration: 216 of 277\ttrain_loss: 0.9000\n",
      "Iteration: 218 of 277\ttrain_loss: 0.9429\n",
      "Iteration: 220 of 277\ttrain_loss: 1.2668\n",
      "Iteration: 222 of 277\ttrain_loss: 1.0121\n",
      "Iteration: 224 of 277\ttrain_loss: 0.9715\n",
      "Iteration: 226 of 277\ttrain_loss: 0.9756\n",
      "Iteration: 228 of 277\ttrain_loss: 0.9409\n",
      "Iteration: 230 of 277\ttrain_loss: 0.9857\n",
      "Iteration: 232 of 277\ttrain_loss: 1.0161\n",
      "Iteration: 234 of 277\ttrain_loss: 0.9882\n",
      "Iteration: 236 of 277\ttrain_loss: 1.0001\n",
      "Iteration: 238 of 277\ttrain_loss: 1.0511\n",
      "Iteration: 240 of 277\ttrain_loss: 0.9158\n",
      "Iteration: 242 of 277\ttrain_loss: 0.9865\n",
      "Iteration: 244 of 277\ttrain_loss: 0.9547\n",
      "Iteration: 246 of 277\ttrain_loss: 0.9437\n",
      "Iteration: 248 of 277\ttrain_loss: 0.9086\n",
      "Iteration: 250 of 277\ttrain_loss: 0.8261\n",
      "Iteration: 252 of 277\ttrain_loss: 1.0454\n",
      "Iteration: 254 of 277\ttrain_loss: 1.0405\n",
      "Iteration: 256 of 277\ttrain_loss: 0.9338\n",
      "Iteration: 258 of 277\ttrain_loss: 1.0687\n",
      "Iteration: 260 of 277\ttrain_loss: 1.0764\n",
      "Iteration: 262 of 277\ttrain_loss: 0.8947\n",
      "Iteration: 264 of 277\ttrain_loss: 1.0780\n",
      "Iteration: 266 of 277\ttrain_loss: 1.1242\n",
      "Iteration: 268 of 277\ttrain_loss: 1.0146\n",
      "Iteration: 270 of 277\ttrain_loss: 0.9130\n",
      "Iteration: 272 of 277\ttrain_loss: 0.9375\n",
      "Iteration: 274 of 277\ttrain_loss: 0.9426\n",
      "Iteration: 276 of 277\ttrain_loss: 0.8992\n",
      "Iteration: 277 of 277\ttrain_loss: 0.9562\n",
      "Average Score for this Epoch: 1.0027382373809814\n",
      "Eval_loss: 2.582105875015259\n",
      "\n",
      "--- new best score ---\n",
      "\n",
      "\n",
      "-------------------- Epoch 36 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 0.9738\n",
      "Iteration: 2 of 277\ttrain_loss: 1.1053\n",
      "Iteration: 4 of 277\ttrain_loss: 0.8330\n",
      "Iteration: 6 of 277\ttrain_loss: 0.8461\n",
      "Iteration: 8 of 277\ttrain_loss: 0.9876\n",
      "Iteration: 10 of 277\ttrain_loss: 0.9068\n",
      "Iteration: 12 of 277\ttrain_loss: 0.9331\n",
      "Iteration: 14 of 277\ttrain_loss: 0.9462\n",
      "Iteration: 16 of 277\ttrain_loss: 1.1024\n",
      "Iteration: 18 of 277\ttrain_loss: 0.9981\n",
      "Iteration: 20 of 277\ttrain_loss: 1.0886\n",
      "Iteration: 22 of 277\ttrain_loss: 1.0943\n",
      "Iteration: 24 of 277\ttrain_loss: 0.9385\n",
      "Iteration: 26 of 277\ttrain_loss: 1.0107\n",
      "Iteration: 28 of 277\ttrain_loss: 0.9525\n",
      "Iteration: 30 of 277\ttrain_loss: 1.0123\n",
      "Iteration: 32 of 277\ttrain_loss: 0.9474\n",
      "Iteration: 34 of 277\ttrain_loss: 1.0879\n",
      "Iteration: 36 of 277\ttrain_loss: 0.9957\n",
      "Iteration: 38 of 277\ttrain_loss: 0.9824\n",
      "Iteration: 40 of 277\ttrain_loss: 1.0183\n",
      "Iteration: 42 of 277\ttrain_loss: 0.9879\n",
      "Iteration: 44 of 277\ttrain_loss: 1.0040\n",
      "Iteration: 46 of 277\ttrain_loss: 0.9098\n",
      "Iteration: 48 of 277\ttrain_loss: 1.2018\n",
      "Iteration: 50 of 277\ttrain_loss: 0.9390\n",
      "Iteration: 52 of 277\ttrain_loss: 0.8813\n",
      "Iteration: 54 of 277\ttrain_loss: 0.9295\n",
      "Iteration: 56 of 277\ttrain_loss: 0.9536\n",
      "Iteration: 58 of 277\ttrain_loss: 1.0320\n",
      "Iteration: 60 of 277\ttrain_loss: 1.4628\n",
      "Iteration: 62 of 277\ttrain_loss: 0.9995\n",
      "Iteration: 64 of 277\ttrain_loss: 0.9247\n",
      "Iteration: 66 of 277\ttrain_loss: 1.0888\n",
      "Iteration: 68 of 277\ttrain_loss: 1.0488\n",
      "Iteration: 70 of 277\ttrain_loss: 0.9106\n",
      "Iteration: 72 of 277\ttrain_loss: 1.0508\n",
      "Iteration: 74 of 277\ttrain_loss: 0.9251\n",
      "Iteration: 76 of 277\ttrain_loss: 1.0857\n",
      "Iteration: 78 of 277\ttrain_loss: 0.9834\n",
      "Iteration: 80 of 277\ttrain_loss: 0.9764\n",
      "Iteration: 82 of 277\ttrain_loss: 0.9221\n",
      "Iteration: 84 of 277\ttrain_loss: 1.1403\n",
      "Iteration: 86 of 277\ttrain_loss: 0.9521\n",
      "Iteration: 88 of 277\ttrain_loss: 0.9872\n",
      "Iteration: 90 of 277\ttrain_loss: 1.0469\n",
      "Iteration: 92 of 277\ttrain_loss: 1.2691\n",
      "Iteration: 94 of 277\ttrain_loss: 1.0066\n",
      "Iteration: 96 of 277\ttrain_loss: 1.0177\n",
      "Iteration: 98 of 277\ttrain_loss: 1.0627\n",
      "Iteration: 100 of 277\ttrain_loss: 0.9493\n",
      "Iteration: 102 of 277\ttrain_loss: 0.9875\n",
      "Iteration: 104 of 277\ttrain_loss: 0.9639\n",
      "Iteration: 106 of 277\ttrain_loss: 1.0070\n",
      "Iteration: 108 of 277\ttrain_loss: 0.8851\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0135\n",
      "Iteration: 112 of 277\ttrain_loss: 0.9157\n",
      "Iteration: 114 of 277\ttrain_loss: 0.9772\n",
      "Iteration: 116 of 277\ttrain_loss: 1.0817\n",
      "Iteration: 118 of 277\ttrain_loss: 1.0196\n",
      "Iteration: 120 of 277\ttrain_loss: 1.0306\n",
      "Iteration: 122 of 277\ttrain_loss: 1.0092\n",
      "Iteration: 124 of 277\ttrain_loss: 0.9848\n",
      "Iteration: 126 of 277\ttrain_loss: 0.9753\n",
      "Iteration: 128 of 277\ttrain_loss: 0.9238\n",
      "Iteration: 130 of 277\ttrain_loss: 0.9269\n",
      "Iteration: 132 of 277\ttrain_loss: 0.9673\n",
      "Iteration: 134 of 277\ttrain_loss: 1.0016\n",
      "Iteration: 136 of 277\ttrain_loss: 0.8850\n",
      "Iteration: 138 of 277\ttrain_loss: 0.9437\n",
      "Iteration: 140 of 277\ttrain_loss: 1.0435\n",
      "Iteration: 142 of 277\ttrain_loss: 1.0217\n",
      "Iteration: 144 of 277\ttrain_loss: 0.9741\n",
      "Iteration: 146 of 277\ttrain_loss: 1.0420\n",
      "Iteration: 148 of 277\ttrain_loss: 1.1449\n",
      "Iteration: 150 of 277\ttrain_loss: 0.9618\n",
      "Iteration: 152 of 277\ttrain_loss: 1.1990\n",
      "Iteration: 154 of 277\ttrain_loss: 0.9704\n",
      "Iteration: 156 of 277\ttrain_loss: 0.9996\n",
      "Iteration: 158 of 277\ttrain_loss: 1.0095\n",
      "Iteration: 160 of 277\ttrain_loss: 0.9830\n",
      "Iteration: 162 of 277\ttrain_loss: 0.9584\n",
      "Iteration: 164 of 277\ttrain_loss: 1.3073\n",
      "Iteration: 166 of 277\ttrain_loss: 1.0129\n",
      "Iteration: 168 of 277\ttrain_loss: 1.0951\n",
      "Iteration: 170 of 277\ttrain_loss: 1.0556\n",
      "Iteration: 172 of 277\ttrain_loss: 0.9793\n",
      "Iteration: 174 of 277\ttrain_loss: 0.9230\n",
      "Iteration: 176 of 277\ttrain_loss: 1.2138\n",
      "Iteration: 178 of 277\ttrain_loss: 1.2102\n",
      "Iteration: 180 of 277\ttrain_loss: 0.9817\n",
      "Iteration: 182 of 277\ttrain_loss: 1.1128\n",
      "Iteration: 184 of 277\ttrain_loss: 1.0315\n",
      "Iteration: 186 of 277\ttrain_loss: 1.2302\n",
      "Iteration: 188 of 277\ttrain_loss: 1.0157\n",
      "Iteration: 190 of 277\ttrain_loss: 0.9241\n",
      "Iteration: 192 of 277\ttrain_loss: 1.0695\n",
      "Iteration: 194 of 277\ttrain_loss: 1.1491\n",
      "Iteration: 196 of 277\ttrain_loss: 1.0671\n",
      "Iteration: 198 of 277\ttrain_loss: 1.0883\n",
      "Iteration: 200 of 277\ttrain_loss: 1.1258\n",
      "Iteration: 202 of 277\ttrain_loss: 0.9292\n",
      "Iteration: 204 of 277\ttrain_loss: 1.1551\n",
      "Iteration: 206 of 277\ttrain_loss: 1.1326\n",
      "Iteration: 208 of 277\ttrain_loss: 0.9568\n",
      "Iteration: 210 of 277\ttrain_loss: 1.0464\n",
      "Iteration: 212 of 277\ttrain_loss: 0.9835\n",
      "Iteration: 214 of 277\ttrain_loss: 1.0606\n",
      "Iteration: 216 of 277\ttrain_loss: 0.9957\n",
      "Iteration: 218 of 277\ttrain_loss: 1.2351\n",
      "Iteration: 220 of 277\ttrain_loss: 0.9854\n",
      "Iteration: 222 of 277\ttrain_loss: 1.1314\n",
      "Iteration: 224 of 277\ttrain_loss: 0.9025\n",
      "Iteration: 226 of 277\ttrain_loss: 1.1341\n",
      "Iteration: 228 of 277\ttrain_loss: 1.0919\n",
      "Iteration: 230 of 277\ttrain_loss: 1.0541\n",
      "Iteration: 232 of 277\ttrain_loss: 1.0498\n",
      "Iteration: 234 of 277\ttrain_loss: 1.2063\n",
      "Iteration: 236 of 277\ttrain_loss: 1.1849\n",
      "Iteration: 238 of 277\ttrain_loss: 1.0096\n",
      "Iteration: 240 of 277\ttrain_loss: 0.9852\n",
      "Iteration: 242 of 277\ttrain_loss: 1.0856\n",
      "Iteration: 244 of 277\ttrain_loss: 1.3213\n",
      "Iteration: 246 of 277\ttrain_loss: 1.1007\n",
      "Iteration: 248 of 277\ttrain_loss: 1.2546\n",
      "Iteration: 250 of 277\ttrain_loss: 1.1175\n",
      "Iteration: 252 of 277\ttrain_loss: 1.1049\n",
      "Iteration: 254 of 277\ttrain_loss: 0.9991\n",
      "Iteration: 256 of 277\ttrain_loss: 0.9032\n",
      "Iteration: 258 of 277\ttrain_loss: 0.9858\n",
      "Iteration: 260 of 277\ttrain_loss: 0.9028\n",
      "Iteration: 262 of 277\ttrain_loss: 1.1565\n",
      "Iteration: 264 of 277\ttrain_loss: 1.0666\n",
      "Iteration: 266 of 277\ttrain_loss: 1.1849\n",
      "Iteration: 268 of 277\ttrain_loss: 1.1139\n",
      "Iteration: 270 of 277\ttrain_loss: 1.2538\n",
      "Iteration: 272 of 277\ttrain_loss: 1.0877\n",
      "Iteration: 274 of 277\ttrain_loss: 1.0142\n",
      "Iteration: 276 of 277\ttrain_loss: 1.1542\n",
      "Iteration: 277 of 277\ttrain_loss: 1.0916\n",
      "Average Score for this Epoch: 1.0372806787490845\n",
      "Eval_loss: 2.621407985687256\n",
      "\n",
      "-------------------- Epoch 37 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 0.9958\n",
      "Iteration: 2 of 277\ttrain_loss: 1.2664\n",
      "Iteration: 4 of 277\ttrain_loss: 0.9707\n",
      "Iteration: 6 of 277\ttrain_loss: 0.8815\n",
      "Iteration: 8 of 277\ttrain_loss: 0.9368\n",
      "Iteration: 10 of 277\ttrain_loss: 1.0826\n",
      "Iteration: 12 of 277\ttrain_loss: 0.9622\n",
      "Iteration: 14 of 277\ttrain_loss: 0.9929\n",
      "Iteration: 16 of 277\ttrain_loss: 0.9854\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1092\n",
      "Iteration: 20 of 277\ttrain_loss: 0.9477\n",
      "Iteration: 22 of 277\ttrain_loss: 0.9037\n",
      "Iteration: 24 of 277\ttrain_loss: 1.0558\n",
      "Iteration: 26 of 277\ttrain_loss: 1.1643\n",
      "Iteration: 28 of 277\ttrain_loss: 0.9465\n",
      "Iteration: 30 of 277\ttrain_loss: 0.9452\n",
      "Iteration: 32 of 277\ttrain_loss: 0.9503\n",
      "Iteration: 34 of 277\ttrain_loss: 0.9621\n",
      "Iteration: 36 of 277\ttrain_loss: 1.1847\n",
      "Iteration: 38 of 277\ttrain_loss: 1.0761\n",
      "Iteration: 40 of 277\ttrain_loss: 1.0606\n",
      "Iteration: 42 of 277\ttrain_loss: 1.3121\n",
      "Iteration: 44 of 277\ttrain_loss: 0.9391\n",
      "Iteration: 46 of 277\ttrain_loss: 1.1615\n",
      "Iteration: 48 of 277\ttrain_loss: 0.9090\n",
      "Iteration: 50 of 277\ttrain_loss: 0.9586\n",
      "Iteration: 52 of 277\ttrain_loss: 1.0812\n",
      "Iteration: 54 of 277\ttrain_loss: 1.0482\n",
      "Iteration: 56 of 277\ttrain_loss: 1.1701\n",
      "Iteration: 58 of 277\ttrain_loss: 1.0212\n",
      "Iteration: 60 of 277\ttrain_loss: 0.9280\n",
      "Iteration: 62 of 277\ttrain_loss: 1.1087\n",
      "Iteration: 64 of 277\ttrain_loss: 0.8974\n",
      "Iteration: 66 of 277\ttrain_loss: 1.1532\n",
      "Iteration: 68 of 277\ttrain_loss: 1.0981\n",
      "Iteration: 70 of 277\ttrain_loss: 1.0224\n",
      "Iteration: 72 of 277\ttrain_loss: 0.9011\n",
      "Iteration: 74 of 277\ttrain_loss: 0.9979\n",
      "Iteration: 76 of 277\ttrain_loss: 0.9763\n",
      "Iteration: 78 of 277\ttrain_loss: 1.0695\n",
      "Iteration: 80 of 277\ttrain_loss: 1.0813\n",
      "Iteration: 82 of 277\ttrain_loss: 1.0120\n",
      "Iteration: 84 of 277\ttrain_loss: 0.9973\n",
      "Iteration: 86 of 277\ttrain_loss: 0.9142\n",
      "Iteration: 88 of 277\ttrain_loss: 1.0692\n",
      "Iteration: 90 of 277\ttrain_loss: 0.9470\n",
      "Iteration: 92 of 277\ttrain_loss: 1.1324\n",
      "Iteration: 94 of 277\ttrain_loss: 1.0511\n",
      "Iteration: 96 of 277\ttrain_loss: 0.9393\n",
      "Iteration: 98 of 277\ttrain_loss: 1.1264\n",
      "Iteration: 100 of 277\ttrain_loss: 1.2035\n",
      "Iteration: 102 of 277\ttrain_loss: 0.9517\n",
      "Iteration: 104 of 277\ttrain_loss: 1.0214\n",
      "Iteration: 106 of 277\ttrain_loss: 1.2740\n",
      "Iteration: 108 of 277\ttrain_loss: 1.0683\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0810\n",
      "Iteration: 112 of 277\ttrain_loss: 1.1300\n",
      "Iteration: 114 of 277\ttrain_loss: 1.0091\n",
      "Iteration: 116 of 277\ttrain_loss: 1.1246\n",
      "Iteration: 118 of 277\ttrain_loss: 1.0578\n",
      "Iteration: 120 of 277\ttrain_loss: 0.9936\n",
      "Iteration: 122 of 277\ttrain_loss: 1.0452\n",
      "Iteration: 124 of 277\ttrain_loss: 1.0928\n",
      "Iteration: 126 of 277\ttrain_loss: 1.0763\n",
      "Iteration: 128 of 277\ttrain_loss: 1.0937\n",
      "Iteration: 130 of 277\ttrain_loss: 1.0765\n",
      "Iteration: 132 of 277\ttrain_loss: 1.1263\n",
      "Iteration: 134 of 277\ttrain_loss: 1.2261\n",
      "Iteration: 136 of 277\ttrain_loss: 0.9775\n",
      "Iteration: 138 of 277\ttrain_loss: 1.2275\n",
      "Iteration: 140 of 277\ttrain_loss: 1.3012\n",
      "Iteration: 142 of 277\ttrain_loss: 1.0588\n",
      "Iteration: 144 of 277\ttrain_loss: 1.2621\n",
      "Iteration: 146 of 277\ttrain_loss: 1.1197\n",
      "Iteration: 148 of 277\ttrain_loss: 1.0520\n",
      "Iteration: 150 of 277\ttrain_loss: 1.2419\n",
      "Iteration: 152 of 277\ttrain_loss: 1.1566\n",
      "Iteration: 154 of 277\ttrain_loss: 1.1381\n",
      "Iteration: 156 of 277\ttrain_loss: 0.9524\n",
      "Iteration: 158 of 277\ttrain_loss: 1.1590\n",
      "Iteration: 160 of 277\ttrain_loss: 1.1719\n",
      "Iteration: 162 of 277\ttrain_loss: 1.0078\n",
      "Iteration: 164 of 277\ttrain_loss: 1.0449\n",
      "Iteration: 166 of 277\ttrain_loss: 1.2719\n",
      "Iteration: 168 of 277\ttrain_loss: 1.2976\n",
      "Iteration: 170 of 277\ttrain_loss: 1.1508\n",
      "Iteration: 172 of 277\ttrain_loss: 1.0205\n",
      "Iteration: 174 of 277\ttrain_loss: 1.0096\n",
      "Iteration: 176 of 277\ttrain_loss: 1.1102\n",
      "Iteration: 178 of 277\ttrain_loss: 1.1070\n",
      "Iteration: 180 of 277\ttrain_loss: 1.0909\n",
      "Iteration: 182 of 277\ttrain_loss: 1.1981\n",
      "Iteration: 184 of 277\ttrain_loss: 1.1350\n",
      "Iteration: 186 of 277\ttrain_loss: 1.1887\n",
      "Iteration: 188 of 277\ttrain_loss: 1.1356\n",
      "Iteration: 190 of 277\ttrain_loss: 1.2986\n",
      "Iteration: 192 of 277\ttrain_loss: 1.0946\n",
      "Iteration: 194 of 277\ttrain_loss: 1.1353\n",
      "Iteration: 196 of 277\ttrain_loss: 1.1855\n",
      "Iteration: 198 of 277\ttrain_loss: 0.9905\n",
      "Iteration: 200 of 277\ttrain_loss: 1.1206\n",
      "Iteration: 202 of 277\ttrain_loss: 1.0642\n",
      "Iteration: 204 of 277\ttrain_loss: 1.3007\n",
      "Iteration: 206 of 277\ttrain_loss: 1.1969\n",
      "Iteration: 208 of 277\ttrain_loss: 1.1024\n",
      "Iteration: 210 of 277\ttrain_loss: 1.2819\n",
      "Iteration: 212 of 277\ttrain_loss: 0.9855\n",
      "Iteration: 214 of 277\ttrain_loss: 1.0894\n",
      "Iteration: 216 of 277\ttrain_loss: 1.1875\n",
      "Iteration: 218 of 277\ttrain_loss: 1.0435\n",
      "Iteration: 220 of 277\ttrain_loss: 1.0743\n",
      "Iteration: 222 of 277\ttrain_loss: 1.1937\n",
      "Iteration: 224 of 277\ttrain_loss: 1.2405\n",
      "Iteration: 226 of 277\ttrain_loss: 1.1570\n",
      "Iteration: 228 of 277\ttrain_loss: 1.3445\n",
      "Iteration: 230 of 277\ttrain_loss: 1.0499\n",
      "Iteration: 232 of 277\ttrain_loss: 1.2719\n",
      "Iteration: 234 of 277\ttrain_loss: 1.0836\n",
      "Iteration: 236 of 277\ttrain_loss: 1.1349\n",
      "Iteration: 238 of 277\ttrain_loss: 1.1093\n",
      "Iteration: 240 of 277\ttrain_loss: 1.1392\n",
      "Iteration: 242 of 277\ttrain_loss: 1.1258\n",
      "Iteration: 244 of 277\ttrain_loss: 1.0175\n",
      "Iteration: 246 of 277\ttrain_loss: 1.2542\n",
      "Iteration: 248 of 277\ttrain_loss: 1.0664\n",
      "Iteration: 250 of 277\ttrain_loss: 1.0436\n",
      "Iteration: 252 of 277\ttrain_loss: 1.1461\n",
      "Iteration: 254 of 277\ttrain_loss: 1.2457\n",
      "Iteration: 256 of 277\ttrain_loss: 1.3134\n",
      "Iteration: 258 of 277\ttrain_loss: 1.3090\n",
      "Iteration: 260 of 277\ttrain_loss: 1.2784\n",
      "Iteration: 262 of 277\ttrain_loss: 1.0811\n",
      "Iteration: 264 of 277\ttrain_loss: 1.1261\n",
      "Iteration: 266 of 277\ttrain_loss: 1.0703\n",
      "Iteration: 268 of 277\ttrain_loss: 1.0927\n",
      "Iteration: 270 of 277\ttrain_loss: 1.1361\n",
      "Iteration: 272 of 277\ttrain_loss: 1.1761\n",
      "Iteration: 274 of 277\ttrain_loss: 1.0805\n",
      "Iteration: 276 of 277\ttrain_loss: 1.2911\n",
      "Iteration: 277 of 277\ttrain_loss: 1.0102\n",
      "Average Score for this Epoch: 1.0991618633270264\n",
      "Eval_loss: 2.7038278579711914\n",
      "\n",
      "-------------------- Epoch 38 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 1.0741\n",
      "Iteration: 2 of 277\ttrain_loss: 1.1100\n",
      "Iteration: 4 of 277\ttrain_loss: 0.9541\n",
      "Iteration: 6 of 277\ttrain_loss: 0.9883\n",
      "Iteration: 8 of 277\ttrain_loss: 0.9913\n",
      "Iteration: 10 of 277\ttrain_loss: 0.9088\n",
      "Iteration: 12 of 277\ttrain_loss: 1.0395\n",
      "Iteration: 14 of 277\ttrain_loss: 1.0283\n",
      "Iteration: 16 of 277\ttrain_loss: 1.0246\n",
      "Iteration: 18 of 277\ttrain_loss: 1.0576\n",
      "Iteration: 20 of 277\ttrain_loss: 1.0502\n",
      "Iteration: 22 of 277\ttrain_loss: 1.0175\n",
      "Iteration: 24 of 277\ttrain_loss: 1.1729\n",
      "Iteration: 26 of 277\ttrain_loss: 1.1457\n",
      "Iteration: 28 of 277\ttrain_loss: 1.1799\n",
      "Iteration: 30 of 277\ttrain_loss: 1.0745\n",
      "Iteration: 32 of 277\ttrain_loss: 0.9795\n",
      "Iteration: 34 of 277\ttrain_loss: 1.0264\n",
      "Iteration: 36 of 277\ttrain_loss: 0.9113\n",
      "Iteration: 38 of 277\ttrain_loss: 1.1167\n",
      "Iteration: 40 of 277\ttrain_loss: 0.8900\n",
      "Iteration: 42 of 277\ttrain_loss: 0.9861\n",
      "Iteration: 44 of 277\ttrain_loss: 0.9081\n",
      "Iteration: 46 of 277\ttrain_loss: 1.0470\n",
      "Iteration: 48 of 277\ttrain_loss: 1.0980\n",
      "Iteration: 50 of 277\ttrain_loss: 1.1097\n",
      "Iteration: 52 of 277\ttrain_loss: 0.9943\n",
      "Iteration: 54 of 277\ttrain_loss: 1.0281\n",
      "Iteration: 56 of 277\ttrain_loss: 1.0522\n",
      "Iteration: 58 of 277\ttrain_loss: 1.0039\n",
      "Iteration: 60 of 277\ttrain_loss: 1.2608\n",
      "Iteration: 62 of 277\ttrain_loss: 1.1132\n",
      "Iteration: 64 of 277\ttrain_loss: 1.0817\n",
      "Iteration: 66 of 277\ttrain_loss: 1.2188\n",
      "Iteration: 68 of 277\ttrain_loss: 1.1675\n",
      "Iteration: 70 of 277\ttrain_loss: 0.9409\n",
      "Iteration: 72 of 277\ttrain_loss: 1.0572\n",
      "Iteration: 74 of 277\ttrain_loss: 0.9520\n",
      "Iteration: 76 of 277\ttrain_loss: 1.1676\n",
      "Iteration: 78 of 277\ttrain_loss: 1.0707\n",
      "Iteration: 80 of 277\ttrain_loss: 1.1194\n",
      "Iteration: 82 of 277\ttrain_loss: 0.9243\n",
      "Iteration: 84 of 277\ttrain_loss: 1.0827\n",
      "Iteration: 86 of 277\ttrain_loss: 1.0554\n",
      "Iteration: 88 of 277\ttrain_loss: 1.0497\n",
      "Iteration: 90 of 277\ttrain_loss: 1.1142\n",
      "Iteration: 92 of 277\ttrain_loss: 0.9580\n",
      "Iteration: 94 of 277\ttrain_loss: 1.1312\n",
      "Iteration: 96 of 277\ttrain_loss: 1.0210\n",
      "Iteration: 98 of 277\ttrain_loss: 1.1708\n",
      "Iteration: 100 of 277\ttrain_loss: 1.1177\n",
      "Iteration: 102 of 277\ttrain_loss: 1.0134\n",
      "Iteration: 104 of 277\ttrain_loss: 1.0811\n",
      "Iteration: 106 of 277\ttrain_loss: 1.1243\n",
      "Iteration: 108 of 277\ttrain_loss: 1.0639\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0355\n",
      "Iteration: 112 of 277\ttrain_loss: 1.3219\n",
      "Iteration: 114 of 277\ttrain_loss: 1.0152\n",
      "Iteration: 116 of 277\ttrain_loss: 1.0380\n",
      "Iteration: 118 of 277\ttrain_loss: 1.0212\n",
      "Iteration: 120 of 277\ttrain_loss: 1.0444\n",
      "Iteration: 122 of 277\ttrain_loss: 1.0635\n",
      "Iteration: 124 of 277\ttrain_loss: 1.1362\n",
      "Iteration: 126 of 277\ttrain_loss: 0.9684\n",
      "Iteration: 128 of 277\ttrain_loss: 1.0337\n",
      "Iteration: 130 of 277\ttrain_loss: 1.0629\n",
      "Iteration: 132 of 277\ttrain_loss: 1.0770\n",
      "Iteration: 134 of 277\ttrain_loss: 1.1154\n",
      "Iteration: 136 of 277\ttrain_loss: 1.1075\n",
      "Iteration: 138 of 277\ttrain_loss: 1.1098\n",
      "Iteration: 140 of 277\ttrain_loss: 1.0060\n",
      "Iteration: 142 of 277\ttrain_loss: 1.3105\n",
      "Iteration: 144 of 277\ttrain_loss: 1.1277\n",
      "Iteration: 146 of 277\ttrain_loss: 1.0419\n",
      "Iteration: 148 of 277\ttrain_loss: 1.1204\n",
      "Iteration: 150 of 277\ttrain_loss: 1.1243\n",
      "Iteration: 152 of 277\ttrain_loss: 1.1160\n",
      "Iteration: 154 of 277\ttrain_loss: 1.0416\n",
      "Iteration: 156 of 277\ttrain_loss: 1.3549\n",
      "Iteration: 158 of 277\ttrain_loss: 1.0480\n",
      "Iteration: 160 of 277\ttrain_loss: 1.0479\n",
      "Iteration: 162 of 277\ttrain_loss: 1.1462\n",
      "Iteration: 164 of 277\ttrain_loss: 1.1930\n",
      "Iteration: 166 of 277\ttrain_loss: 1.2697\n",
      "Iteration: 168 of 277\ttrain_loss: 1.1084\n",
      "Iteration: 170 of 277\ttrain_loss: 1.1317\n",
      "Iteration: 172 of 277\ttrain_loss: 0.9860\n",
      "Iteration: 174 of 277\ttrain_loss: 1.0241\n",
      "Iteration: 176 of 277\ttrain_loss: 1.0625\n",
      "Iteration: 178 of 277\ttrain_loss: 1.2485\n",
      "Iteration: 180 of 277\ttrain_loss: 1.0613\n",
      "Iteration: 182 of 277\ttrain_loss: 1.2951\n",
      "Iteration: 184 of 277\ttrain_loss: 1.0698\n",
      "Iteration: 186 of 277\ttrain_loss: 1.1624\n",
      "Iteration: 188 of 277\ttrain_loss: 1.1452\n",
      "Iteration: 190 of 277\ttrain_loss: 0.8874\n",
      "Iteration: 192 of 277\ttrain_loss: 1.0406\n",
      "Iteration: 194 of 277\ttrain_loss: 1.2053\n",
      "Iteration: 196 of 277\ttrain_loss: 1.0700\n",
      "Iteration: 198 of 277\ttrain_loss: 1.1975\n",
      "Iteration: 200 of 277\ttrain_loss: 1.2774\n",
      "Iteration: 202 of 277\ttrain_loss: 1.1206\n",
      "Iteration: 204 of 277\ttrain_loss: 1.0356\n",
      "Iteration: 206 of 277\ttrain_loss: 1.2797\n",
      "Iteration: 208 of 277\ttrain_loss: 1.0431\n",
      "Iteration: 210 of 277\ttrain_loss: 1.2363\n",
      "Iteration: 212 of 277\ttrain_loss: 1.0445\n",
      "Iteration: 214 of 277\ttrain_loss: 1.0354\n",
      "Iteration: 216 of 277\ttrain_loss: 1.1814\n",
      "Iteration: 218 of 277\ttrain_loss: 1.0615\n",
      "Iteration: 220 of 277\ttrain_loss: 1.0325\n",
      "Iteration: 222 of 277\ttrain_loss: 0.9778\n",
      "Iteration: 224 of 277\ttrain_loss: 0.9980\n",
      "Iteration: 226 of 277\ttrain_loss: 1.1861\n",
      "Iteration: 228 of 277\ttrain_loss: 1.1277\n",
      "Iteration: 230 of 277\ttrain_loss: 1.0629\n",
      "Iteration: 232 of 277\ttrain_loss: 0.9326\n",
      "Iteration: 234 of 277\ttrain_loss: 1.0220\n",
      "Iteration: 236 of 277\ttrain_loss: 1.0519\n",
      "Iteration: 238 of 277\ttrain_loss: 1.0470\n",
      "Iteration: 240 of 277\ttrain_loss: 1.0640\n",
      "Iteration: 242 of 277\ttrain_loss: 1.0549\n",
      "Iteration: 244 of 277\ttrain_loss: 1.1700\n",
      "Iteration: 246 of 277\ttrain_loss: 1.0868\n",
      "Iteration: 248 of 277\ttrain_loss: 0.9545\n",
      "Iteration: 250 of 277\ttrain_loss: 1.1946\n",
      "Iteration: 252 of 277\ttrain_loss: 1.0074\n",
      "Iteration: 254 of 277\ttrain_loss: 1.3031\n",
      "Iteration: 256 of 277\ttrain_loss: 1.0474\n",
      "Iteration: 258 of 277\ttrain_loss: 1.2080\n",
      "Iteration: 260 of 277\ttrain_loss: 1.0326\n",
      "Iteration: 262 of 277\ttrain_loss: 1.0828\n",
      "Iteration: 264 of 277\ttrain_loss: 1.2595\n",
      "Iteration: 266 of 277\ttrain_loss: 0.9772\n",
      "Iteration: 268 of 277\ttrain_loss: 1.1260\n",
      "Iteration: 270 of 277\ttrain_loss: 1.1412\n",
      "Iteration: 272 of 277\ttrain_loss: 1.3325\n",
      "Iteration: 274 of 277\ttrain_loss: 1.0969\n",
      "Iteration: 276 of 277\ttrain_loss: 1.0215\n",
      "Iteration: 277 of 277\ttrain_loss: 0.9188\n",
      "Average Score for this Epoch: 1.0934844017028809\n",
      "Eval_loss: 2.6348798274993896\n",
      "\n",
      "-------------------- Epoch 39 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 0.9663\n",
      "Iteration: 2 of 277\ttrain_loss: 1.0632\n",
      "Iteration: 4 of 277\ttrain_loss: 0.9799\n",
      "Iteration: 6 of 277\ttrain_loss: 0.9093\n",
      "Iteration: 8 of 277\ttrain_loss: 1.0331\n",
      "Iteration: 10 of 277\ttrain_loss: 1.0499\n",
      "Iteration: 12 of 277\ttrain_loss: 0.9914\n",
      "Iteration: 14 of 277\ttrain_loss: 1.1439\n",
      "Iteration: 16 of 277\ttrain_loss: 1.0548\n",
      "Iteration: 18 of 277\ttrain_loss: 0.9179\n",
      "Iteration: 20 of 277\ttrain_loss: 0.9948\n",
      "Iteration: 22 of 277\ttrain_loss: 0.9120\n",
      "Iteration: 24 of 277\ttrain_loss: 0.9377\n",
      "Iteration: 26 of 277\ttrain_loss: 1.1128\n",
      "Iteration: 28 of 277\ttrain_loss: 1.0649\n",
      "Iteration: 30 of 277\ttrain_loss: 0.9855\n",
      "Iteration: 32 of 277\ttrain_loss: 0.9269\n",
      "Iteration: 34 of 277\ttrain_loss: 1.0469\n",
      "Iteration: 36 of 277\ttrain_loss: 0.8694\n",
      "Iteration: 38 of 277\ttrain_loss: 0.9849\n",
      "Iteration: 40 of 277\ttrain_loss: 1.0669\n",
      "Iteration: 42 of 277\ttrain_loss: 1.0319\n",
      "Iteration: 44 of 277\ttrain_loss: 1.0009\n",
      "Iteration: 46 of 277\ttrain_loss: 0.9202\n",
      "Iteration: 48 of 277\ttrain_loss: 0.9700\n",
      "Iteration: 50 of 277\ttrain_loss: 1.0486\n",
      "Iteration: 52 of 277\ttrain_loss: 1.0248\n",
      "Iteration: 54 of 277\ttrain_loss: 0.8970\n",
      "Iteration: 56 of 277\ttrain_loss: 0.9016\n",
      "Iteration: 58 of 277\ttrain_loss: 0.9029\n",
      "Iteration: 60 of 277\ttrain_loss: 1.0469\n",
      "Iteration: 62 of 277\ttrain_loss: 1.0001\n",
      "Iteration: 64 of 277\ttrain_loss: 0.9601\n",
      "Iteration: 66 of 277\ttrain_loss: 0.8908\n",
      "Iteration: 68 of 277\ttrain_loss: 1.0344\n",
      "Iteration: 70 of 277\ttrain_loss: 0.9552\n",
      "Iteration: 72 of 277\ttrain_loss: 1.0748\n",
      "Iteration: 74 of 277\ttrain_loss: 0.9946\n",
      "Iteration: 76 of 277\ttrain_loss: 1.0754\n",
      "Iteration: 78 of 277\ttrain_loss: 1.0471\n",
      "Iteration: 80 of 277\ttrain_loss: 1.0151\n",
      "Iteration: 82 of 277\ttrain_loss: 0.8737\n",
      "Iteration: 84 of 277\ttrain_loss: 1.1484\n",
      "Iteration: 86 of 277\ttrain_loss: 0.8845\n",
      "Iteration: 88 of 277\ttrain_loss: 0.8907\n",
      "Iteration: 90 of 277\ttrain_loss: 0.9637\n",
      "Iteration: 92 of 277\ttrain_loss: 0.9775\n",
      "Iteration: 94 of 277\ttrain_loss: 1.0184\n",
      "Iteration: 96 of 277\ttrain_loss: 0.9732\n",
      "Iteration: 98 of 277\ttrain_loss: 0.9890\n",
      "Iteration: 100 of 277\ttrain_loss: 1.1343\n",
      "Iteration: 102 of 277\ttrain_loss: 1.0372\n",
      "Iteration: 104 of 277\ttrain_loss: 1.0802\n",
      "Iteration: 106 of 277\ttrain_loss: 1.0477\n",
      "Iteration: 108 of 277\ttrain_loss: 0.9668\n",
      "Iteration: 110 of 277\ttrain_loss: 1.0101\n",
      "Iteration: 112 of 277\ttrain_loss: 0.9871\n",
      "Iteration: 114 of 277\ttrain_loss: 1.0267\n",
      "Iteration: 116 of 277\ttrain_loss: 0.9368\n",
      "Iteration: 118 of 277\ttrain_loss: 0.9325\n",
      "Iteration: 120 of 277\ttrain_loss: 1.0253\n",
      "Iteration: 122 of 277\ttrain_loss: 1.0587\n",
      "Iteration: 124 of 277\ttrain_loss: 0.9944\n",
      "Iteration: 126 of 277\ttrain_loss: 1.0986\n",
      "Iteration: 128 of 277\ttrain_loss: 0.8797\n",
      "Iteration: 130 of 277\ttrain_loss: 1.0863\n",
      "Iteration: 132 of 277\ttrain_loss: 1.0836\n",
      "Iteration: 134 of 277\ttrain_loss: 1.0868\n",
      "Iteration: 136 of 277\ttrain_loss: 1.0453\n",
      "Iteration: 138 of 277\ttrain_loss: 0.9829\n",
      "Iteration: 140 of 277\ttrain_loss: 1.2757\n",
      "Iteration: 142 of 277\ttrain_loss: 1.0846\n",
      "Iteration: 144 of 277\ttrain_loss: 0.9431\n",
      "Iteration: 146 of 277\ttrain_loss: 0.9222\n",
      "Iteration: 148 of 277\ttrain_loss: 1.0402\n",
      "Iteration: 150 of 277\ttrain_loss: 1.0451\n",
      "Iteration: 152 of 277\ttrain_loss: 0.9484\n",
      "Iteration: 154 of 277\ttrain_loss: 1.0474\n",
      "Iteration: 156 of 277\ttrain_loss: 0.9831\n",
      "Iteration: 158 of 277\ttrain_loss: 1.0371\n",
      "Iteration: 160 of 277\ttrain_loss: 1.0532\n",
      "Iteration: 162 of 277\ttrain_loss: 0.7878\n",
      "Iteration: 164 of 277\ttrain_loss: 0.9815\n",
      "Iteration: 166 of 277\ttrain_loss: 1.1836\n",
      "Iteration: 168 of 277\ttrain_loss: 0.9351\n",
      "Iteration: 170 of 277\ttrain_loss: 1.0144\n",
      "Iteration: 172 of 277\ttrain_loss: 1.0811\n",
      "Iteration: 174 of 277\ttrain_loss: 0.9097\n",
      "Iteration: 176 of 277\ttrain_loss: 1.1113\n",
      "Iteration: 178 of 277\ttrain_loss: 0.9908\n",
      "Iteration: 180 of 277\ttrain_loss: 1.1317\n",
      "Iteration: 182 of 277\ttrain_loss: 1.0619\n",
      "Iteration: 184 of 277\ttrain_loss: 1.0479\n",
      "Iteration: 186 of 277\ttrain_loss: 0.9990\n",
      "Iteration: 188 of 277\ttrain_loss: 0.9285\n",
      "Iteration: 190 of 277\ttrain_loss: 1.0305\n",
      "Iteration: 192 of 277\ttrain_loss: 0.9858\n",
      "Iteration: 194 of 277\ttrain_loss: 1.0278\n",
      "Iteration: 196 of 277\ttrain_loss: 1.0428\n",
      "Iteration: 198 of 277\ttrain_loss: 1.0613\n",
      "Iteration: 200 of 277\ttrain_loss: 0.9818\n",
      "Iteration: 202 of 277\ttrain_loss: 1.0715\n",
      "Iteration: 204 of 277\ttrain_loss: 1.0025\n",
      "Iteration: 206 of 277\ttrain_loss: 1.2267\n",
      "Iteration: 208 of 277\ttrain_loss: 0.9889\n",
      "Iteration: 210 of 277\ttrain_loss: 0.8845\n",
      "Iteration: 212 of 277\ttrain_loss: 0.8983\n",
      "Iteration: 214 of 277\ttrain_loss: 0.9157\n",
      "Iteration: 216 of 277\ttrain_loss: 1.0637\n",
      "Iteration: 218 of 277\ttrain_loss: 0.9538\n",
      "Iteration: 220 of 277\ttrain_loss: 0.9385\n",
      "Iteration: 222 of 277\ttrain_loss: 0.8923\n",
      "Iteration: 224 of 277\ttrain_loss: 1.0046\n",
      "Iteration: 226 of 277\ttrain_loss: 0.8820\n",
      "Iteration: 228 of 277\ttrain_loss: 1.0012\n",
      "Iteration: 230 of 277\ttrain_loss: 1.0312\n",
      "Iteration: 232 of 277\ttrain_loss: 0.9274\n",
      "Iteration: 234 of 277\ttrain_loss: 1.0717\n",
      "Iteration: 236 of 277\ttrain_loss: 0.8898\n",
      "Iteration: 238 of 277\ttrain_loss: 1.0389\n",
      "Iteration: 240 of 277\ttrain_loss: 0.9826\n",
      "Iteration: 242 of 277\ttrain_loss: 1.0419\n",
      "Iteration: 244 of 277\ttrain_loss: 0.9752\n",
      "Iteration: 246 of 277\ttrain_loss: 1.0448\n",
      "Iteration: 248 of 277\ttrain_loss: 0.9946\n",
      "Iteration: 250 of 277\ttrain_loss: 0.8827\n",
      "Iteration: 252 of 277\ttrain_loss: 1.0147\n",
      "Iteration: 254 of 277\ttrain_loss: 1.0054\n",
      "Iteration: 256 of 277\ttrain_loss: 1.0230\n",
      "Iteration: 258 of 277\ttrain_loss: 1.0129\n",
      "Iteration: 260 of 277\ttrain_loss: 0.9244\n",
      "Iteration: 262 of 277\ttrain_loss: 1.0257\n",
      "Iteration: 264 of 277\ttrain_loss: 1.3875\n",
      "Iteration: 266 of 277\ttrain_loss: 0.9558\n",
      "Iteration: 268 of 277\ttrain_loss: 0.8926\n",
      "Iteration: 270 of 277\ttrain_loss: 0.9787\n",
      "Iteration: 272 of 277\ttrain_loss: 0.9762\n",
      "Iteration: 274 of 277\ttrain_loss: 0.9287\n",
      "Iteration: 276 of 277\ttrain_loss: 1.0183\n",
      "Iteration: 277 of 277\ttrain_loss: 0.8906\n",
      "Average Score for this Epoch: 1.0085657835006714\n",
      "Eval_loss: 2.6295583248138428\n",
      "\n",
      "-------------------- Epoch 40 of 200 --------------------\n",
      "Iteration: 0 of 277\ttrain_loss: 0.9333\n",
      "Iteration: 2 of 277\ttrain_loss: 0.9683\n",
      "Iteration: 4 of 277\ttrain_loss: 0.9596\n",
      "Iteration: 6 of 277\ttrain_loss: 1.1606\n",
      "Iteration: 8 of 277\ttrain_loss: 0.8394\n",
      "Iteration: 10 of 277\ttrain_loss: 1.0189\n",
      "Iteration: 12 of 277\ttrain_loss: 0.9300\n",
      "Iteration: 14 of 277\ttrain_loss: 0.8449\n",
      "Iteration: 16 of 277\ttrain_loss: 0.9221\n",
      "Iteration: 18 of 277\ttrain_loss: 1.1452\n",
      "Iteration: 20 of 277\ttrain_loss: 1.1007\n",
      "Iteration: 22 of 277\ttrain_loss: 1.0877\n",
      "Iteration: 24 of 277\ttrain_loss: 1.0162\n",
      "Iteration: 26 of 277\ttrain_loss: 0.8973\n",
      "Iteration: 28 of 277\ttrain_loss: 0.8954\n",
      "Iteration: 30 of 277\ttrain_loss: 0.8441\n",
      "Iteration: 32 of 277\ttrain_loss: 0.8186\n",
      "Iteration: 34 of 277\ttrain_loss: 0.9476\n",
      "Iteration: 36 of 277\ttrain_loss: 0.8355\n",
      "Iteration: 38 of 277\ttrain_loss: 0.9368\n",
      "Iteration: 40 of 277\ttrain_loss: 0.9759\n",
      "Iteration: 42 of 277\ttrain_loss: 0.9004\n",
      "Iteration: 44 of 277\ttrain_loss: 0.9828\n",
      "Iteration: 46 of 277\ttrain_loss: 0.9444\n",
      "Iteration: 48 of 277\ttrain_loss: 0.9065\n",
      "Iteration: 50 of 277\ttrain_loss: 0.9347\n",
      "Iteration: 52 of 277\ttrain_loss: 0.9738\n",
      "Iteration: 54 of 277\ttrain_loss: 0.9880\n",
      "Iteration: 56 of 277\ttrain_loss: 0.9212\n",
      "Iteration: 58 of 277\ttrain_loss: 0.9572\n",
      "Iteration: 60 of 277\ttrain_loss: 0.7422\n",
      "Iteration: 62 of 277\ttrain_loss: 0.9540\n",
      "Iteration: 64 of 277\ttrain_loss: 0.9785\n",
      "Iteration: 66 of 277\ttrain_loss: 0.8233\n",
      "Iteration: 68 of 277\ttrain_loss: 0.8660\n",
      "Iteration: 70 of 277\ttrain_loss: 1.1025\n",
      "Iteration: 72 of 277\ttrain_loss: 0.8950\n",
      "Iteration: 74 of 277\ttrain_loss: 1.0732\n",
      "Iteration: 76 of 277\ttrain_loss: 0.8493\n",
      "Iteration: 78 of 277\ttrain_loss: 0.9175\n",
      "Iteration: 80 of 277\ttrain_loss: 1.2908\n",
      "Iteration: 82 of 277\ttrain_loss: 1.0473\n",
      "Iteration: 84 of 277\ttrain_loss: 0.9049\n",
      "Iteration: 86 of 277\ttrain_loss: 0.9496\n",
      "Iteration: 88 of 277\ttrain_loss: 0.9230\n",
      "Iteration: 90 of 277\ttrain_loss: 0.9420\n",
      "Iteration: 92 of 277\ttrain_loss: 0.9661\n",
      "Iteration: 94 of 277\ttrain_loss: 0.9427\n",
      "Iteration: 96 of 277\ttrain_loss: 0.9628\n",
      "Iteration: 98 of 277\ttrain_loss: 0.9824\n",
      "Iteration: 100 of 277\ttrain_loss: 0.9236\n",
      "Iteration: 102 of 277\ttrain_loss: 0.9968\n",
      "Iteration: 104 of 277\ttrain_loss: 0.9248\n",
      "Iteration: 106 of 277\ttrain_loss: 0.8555\n",
      "Iteration: 108 of 277\ttrain_loss: 0.9573\n",
      "Iteration: 110 of 277\ttrain_loss: 0.8784\n",
      "Iteration: 112 of 277\ttrain_loss: 0.9027\n",
      "Iteration: 114 of 277\ttrain_loss: 1.0235\n",
      "Iteration: 116 of 277\ttrain_loss: 0.9186\n",
      "Iteration: 118 of 277\ttrain_loss: 0.8536\n",
      "Iteration: 120 of 277\ttrain_loss: 0.9668\n",
      "Iteration: 122 of 277\ttrain_loss: 0.9220\n",
      "Iteration: 124 of 277\ttrain_loss: 0.8978\n",
      "Iteration: 126 of 277\ttrain_loss: 1.1050\n",
      "Iteration: 128 of 277\ttrain_loss: 0.8247\n",
      "Iteration: 130 of 277\ttrain_loss: 0.8964\n",
      "Iteration: 132 of 277\ttrain_loss: 0.9624\n",
      "Iteration: 134 of 277\ttrain_loss: 0.8526\n",
      "Iteration: 136 of 277\ttrain_loss: 0.9780\n",
      "Iteration: 138 of 277\ttrain_loss: 0.8674\n",
      "Iteration: 140 of 277\ttrain_loss: 0.8697\n",
      "Iteration: 142 of 277\ttrain_loss: 0.8567\n",
      "Iteration: 144 of 277\ttrain_loss: 0.9985\n",
      "Iteration: 146 of 277\ttrain_loss: 0.9807\n",
      "Iteration: 148 of 277\ttrain_loss: 0.9797\n",
      "Iteration: 150 of 277\ttrain_loss: 0.8800\n",
      "Iteration: 152 of 277\ttrain_loss: 1.0653\n",
      "Iteration: 154 of 277\ttrain_loss: 0.8411\n",
      "Iteration: 156 of 277\ttrain_loss: 1.1965\n",
      "Iteration: 158 of 277\ttrain_loss: 0.8725\n",
      "Iteration: 160 of 277\ttrain_loss: 1.1026\n",
      "Iteration: 162 of 277\ttrain_loss: 0.9073\n",
      "Iteration: 164 of 277\ttrain_loss: 0.8106\n",
      "Iteration: 166 of 277\ttrain_loss: 1.0320\n",
      "Iteration: 168 of 277\ttrain_loss: 0.9587\n",
      "Iteration: 170 of 277\ttrain_loss: 0.9020\n",
      "Iteration: 172 of 277\ttrain_loss: 0.8591\n",
      "Iteration: 174 of 277\ttrain_loss: 0.9387\n",
      "Iteration: 176 of 277\ttrain_loss: 0.9085\n",
      "Iteration: 178 of 277\ttrain_loss: 0.9685\n",
      "Iteration: 180 of 277\ttrain_loss: 0.9418\n",
      "Iteration: 182 of 277\ttrain_loss: 1.0906\n",
      "Iteration: 184 of 277\ttrain_loss: 0.9909\n",
      "Iteration: 186 of 277\ttrain_loss: 0.8414\n",
      "Iteration: 188 of 277\ttrain_loss: 0.8988\n",
      "Iteration: 190 of 277\ttrain_loss: 0.8275\n",
      "Iteration: 192 of 277\ttrain_loss: 0.8966\n",
      "Iteration: 194 of 277\ttrain_loss: 0.9417\n",
      "Iteration: 196 of 277\ttrain_loss: 1.0090\n",
      "Iteration: 198 of 277\ttrain_loss: 0.9417\n",
      "Iteration: 200 of 277\ttrain_loss: 0.9232\n",
      "Iteration: 202 of 277\ttrain_loss: 0.9328\n",
      "Iteration: 204 of 277\ttrain_loss: 0.8970\n",
      "Iteration: 206 of 277\ttrain_loss: 0.9459\n",
      "Iteration: 208 of 277\ttrain_loss: 0.9304\n",
      "Iteration: 210 of 277\ttrain_loss: 0.8813\n",
      "Iteration: 212 of 277\ttrain_loss: 0.7305\n",
      "Iteration: 214 of 277\ttrain_loss: 1.1012\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "ignored",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-75-b25d31a0c974>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     24\u001b[0m                  \u001b[0mconverted_summaries\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;36m70976\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     25\u001b[0m                  \u001b[0mvalidation_inputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mconverted_texts\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m70976\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 26\u001b[0;31m                  validation_targets=converted_summaries[70976:])\n\u001b[0m\u001b[1;32m     27\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     28\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-74-762b6ec99214>\u001b[0m in \u001b[0;36mtrain\u001b[0;34m(self, inputs, targets, restore_path, validation_inputs, validation_targets)\u001b[0m\n\u001b[1;32m    451\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    452\u001b[0m             \u001b[0;31m# run training epoch\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 453\u001b[0;31m             \u001b[0mscore\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_epoch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtargets\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mepoch\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    454\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    455\u001b[0m             \u001b[0;31m# evaluate model\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-74-762b6ec99214>\u001b[0m in \u001b[0;36mrun_epoch\u001b[0;34m(self, inputs, targets, epoch)\u001b[0m\n\u001b[1;32m    524\u001b[0m                 \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    525\u001b[0m                     _, train_loss = self.sess.run([self.train_op, self.train_loss],\n\u001b[0;32m--> 526\u001b[0;31m                                                   feed_dict=fd)\n\u001b[0m\u001b[1;32m    527\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    528\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mi\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0;36m2\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mnbatches\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m    927\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    928\u001b[0m       result = self._run(None, fetches, feed_dict, options_ptr,\n\u001b[0;32m--> 929\u001b[0;31m                          run_metadata_ptr)\n\u001b[0m\u001b[1;32m    930\u001b[0m       \u001b[0;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    931\u001b[0m         \u001b[0mproto_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_run\u001b[0;34m(self, handle, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m   1150\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mfinal_fetches\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mfinal_targets\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mhandle\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mfeed_dict_tensor\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1151\u001b[0m       results = self._do_run(handle, final_targets, final_fetches,\n\u001b[0;32m-> 1152\u001b[0;31m                              feed_dict_tensor, options, run_metadata)\n\u001b[0m\u001b[1;32m   1153\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1154\u001b[0m       \u001b[0mresults\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_do_run\u001b[0;34m(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m   1326\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mhandle\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1327\u001b[0m       return self._do_call(_run_fn, feeds, fetches, targets, options,\n\u001b[0;32m-> 1328\u001b[0;31m                            run_metadata)\n\u001b[0m\u001b[1;32m   1329\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1330\u001b[0m       \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_do_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_prun_fn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeeds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetches\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_do_call\u001b[0;34m(self, fn, *args)\u001b[0m\n\u001b[1;32m   1332\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_do_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1333\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1334\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1335\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mOpError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1336\u001b[0m       \u001b[0mmessage\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcompat\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mas_text\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_run_fn\u001b[0;34m(feed_dict, fetch_list, target_list, options, run_metadata)\u001b[0m\n\u001b[1;32m   1317\u001b[0m       \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_extend_graph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1318\u001b[0m       return self._call_tf_sessionrun(\n\u001b[0;32m-> 1319\u001b[0;31m           options, feed_dict, fetch_list, target_list, run_metadata)\n\u001b[0m\u001b[1;32m   1320\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1321\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_prun_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m_call_tf_sessionrun\u001b[0;34m(self, options, feed_dict, fetch_list, target_list, run_metadata)\u001b[0m\n\u001b[1;32m   1405\u001b[0m     return tf_session.TF_SessionRun_wrapper(\n\u001b[1;32m   1406\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moptions\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget_list\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1407\u001b[0;31m         run_metadata)\n\u001b[0m\u001b[1;32m   1408\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1409\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_call_tf_sessionprun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "# build graph and train the model \n",
    "reset_graph()\n",
    "summarizer = Summarizer(word2ind,\n",
    "                                   ind2word,\n",
    "                                   save_path='drive/Colab Notebooks/Model 3/my_model',\n",
    "                                   mode='TRAIN',\n",
    "                                   num_layers_encoder = num_layers_encoder,\n",
    "                                   num_layers_decoder = num_layers_decoder,\n",
    "                                   rnn_size_encoder = rnn_size_encoder,\n",
    "                                   rnn_size_decoder = rnn_size_decoder,\n",
    "                                   batch_size = batch_size,\n",
    "                                   clip = clip,\n",
    "                                   keep_probability = keep_probability,\n",
    "                                   learning_rate = learning_rate,\n",
    "                                   max_lr=max_lr,\n",
    "                                   learning_rate_decay_steps = learning_rate_decay_steps,\n",
    "                                   learning_rate_decay = learning_rate_decay,\n",
    "                                   epochs = epochs,\n",
    "                                   pretrained_embeddings_path = pretrained_embeddings_path,\n",
    "                                   use_cyclic_lr = use_cyclic_lr,\n",
    "                                   summary_dir = summary_dir)           \n",
    "\n",
    "summarizer.build_graph()\n",
    "summarizer.train(converted_texts[:70976], \n",
    "                 converted_summaries[:70976],\n",
    "                 validation_inputs=converted_texts[70976:],\n",
    "                 validation_targets=converted_summaries[70976:])\n",
    "\n",
    "\n",
    "# hidden training output.\n",
    "# both train and validation loss decrease nicely."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "U5Hqzvocmw2W"
   },
   "source": [
    "### Inference\n",
    "Now we can use our trained model to create summaries. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 90
    },
    "colab_type": "code",
    "id": "ljN9a1hemw2Y",
    "outputId": "c3fdba46-e256-4067-dfbd-3f3b8af334a6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded pretrained embeddings.\n",
      "Graph built.\n",
      "INFO:tensorflow:Restoring parameters from drive/Colab Notebooks/Model 3/my_model\n",
      "Done.\n"
     ]
    }
   ],
   "source": [
    "reset_graph()\n",
    "summarizer = Summarizer(word2ind,\n",
    "                                   ind2word,\n",
    "                                   'drive/Colab Notebooks/Model 3/my_model',\n",
    "                                   'INFER',\n",
    "                                   num_layers_encoder = num_layers_encoder,\n",
    "                                   num_layers_decoder = num_layers_decoder,\n",
    "                                   batch_size = len(converted_texts[:50]),\n",
    "                                   clip = clip,\n",
    "                                   keep_probability = 1.0,\n",
    "                                   learning_rate = 0.0,\n",
    "                                   beam_width = 5,\n",
    "                                   rnn_size_encoder = rnn_size_encoder,\n",
    "                                   rnn_size_decoder = rnn_size_decoder,\n",
    "                                   inference_targets = True,\n",
    "                                   pretrained_embeddings_path = pretrained_embeddings_path)\n",
    "\n",
    "summarizer.build_graph()\n",
    "preds = summarizer.infer(converted_texts[:50],\n",
    "                         restore_path = 'drive/Colab Notebooks/Model 3/my_model',\n",
    "                         targets = converted_summaries[:50])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 12765
    },
    "colab_type": "code",
    "id": "JtB2kNIWmw2j",
    "outputId": "68bb94d4-7d23-4c3e-bc6b-d38c80992f4e",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "great taffy at a great price there was a wide assortment of yummy taffy delivery was very quick if your a taffy lover this is a deal\n",
      "\n",
      "Actual Summary:\n",
      "great taffy\n",
      "\n",
      "Created Summary:\n",
      "great taffy taffy\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "this taffy is so good it is very soft and chewy the flavors are amazing i would definitely recommend you buying it very satisfying\n",
      "\n",
      "Actual Summary:\n",
      "wonderful tasty taffy\n",
      "\n",
      "Created Summary:\n",
      "delicious taffy taffy\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "right now i m mostly just sprouting this so my cats can eat the grass they love it i rotate it around with wheatgrass and rye too\n",
      "\n",
      "Actual Summary:\n",
      "yay barley\n",
      "\n",
      "Created Summary:\n",
      "yay barley\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "this is a very healthy dog food good for their digestion also good for small puppies my dog eats her required amount at every feeding\n",
      "\n",
      "Actual Summary:\n",
      "healthy dog food\n",
      "\n",
      "Created Summary:\n",
      "healthy healthy food\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "the strawberry twizzlers are my guilty pleasure yummy six pounds will be around for a while with my son and i\n",
      "\n",
      "Actual Summary:\n",
      "strawberry twizzlers yummy\n",
      "\n",
      "Created Summary:\n",
      "the\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i am very satisfied with my twizzler purchase i shared these with others and we have all enjoyed them i will definitely be ordering more\n",
      "\n",
      "Actual Summary:\n",
      "love it\n",
      "\n",
      "Created Summary:\n",
      "love it\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "candy was delivered very fast and was purchased at a reasonable price i was home bound and unable to get to a store so this was perfect for me\n",
      "\n",
      "Actual Summary:\n",
      "home delivered twizlers\n",
      "\n",
      "Created Summary:\n",
      "delivered delivered twizlers\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i can remember buying this candy as a kid and the quality hasn t dropped in all these years still a superb product you won t be disappointed with\n",
      "\n",
      "Actual Summary:\n",
      "delicious product\n",
      "\n",
      "Created Summary:\n",
      "i memories memories\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "this offer is a great price and a great taste thanks amazon for selling this product staral\n",
      "\n",
      "Actual Summary:\n",
      "this is my taste\n",
      "\n",
      "Created Summary:\n",
      "great\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "the flavors are good however i do not see any differce between this and oaker oats brand they are both mushy\n",
      "\n",
      "Actual Summary:\n",
      "mushy\n",
      "\n",
      "Created Summary:\n",
      "good good\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "halloween is over but i sent a bag to my daughters class for her share the chocolate was fresh and enjoyed by many\n",
      "\n",
      "Actual Summary:\n",
      "great deal\n",
      "\n",
      "Created Summary:\n",
      "great\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "no tea flavor at all just whole brunch of artifial flavors it is not returnable i wasted 20 bucks\n",
      "\n",
      "Actual Summary:\n",
      "no tea flavor\n",
      "\n",
      "Created Summary:\n",
      "great tea tea\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "the taste was great but the berries had melted may order again in winter if you order in cold weather you should enjoy flavor\n",
      "\n",
      "Actual Summary:\n",
      "order only in cold weather\n",
      "\n",
      "Created Summary:\n",
      "melted melted\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "great gift for all ages i purchased these giant canes before and the recipients loved them so much they kept them and would not eat them\n",
      "\n",
      "Actual Summary:\n",
      "great\n",
      "\n",
      "Created Summary:\n",
      "great\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "this is great dog food my dog has severs allergies and this brand is the only one that we can feed him\n",
      "\n",
      "Actual Summary:\n",
      "great dog food\n",
      "\n",
      "Created Summary:\n",
      "great dog food\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "this is the same food we get at pet store but it s delivered to my door and for the same price or slightly less\n",
      "\n",
      "Actual Summary:\n",
      "so convenient\n",
      "\n",
      "Created Summary:\n",
      "good food food\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "arrived slightly thawed my parents wouldn t accept it however the company was very helpful and issued a full refund\n",
      "\n",
      "Actual Summary:\n",
      "great support\n",
      "\n",
      "Created Summary:\n",
      "great support\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "this has to be one of the best teas i have ever tasted it s clean bright fresh great delivery again quality just try it\n",
      "\n",
      "Actual Summary:\n",
      "the best tea ever freah bright clean\n",
      "\n",
      "Created Summary:\n",
      "awesome\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i love this tea it helps curb my eating during the day my mom and i have given it all friends to try\n",
      "\n",
      "Actual Summary:\n",
      "wonderful tea\n",
      "\n",
      "Created Summary:\n",
      "wonderful tea\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i have bought allot of different flavors and this happens to be one of my favorites and will be getting more soon\n",
      "\n",
      "Actual Summary:\n",
      "great flavor\n",
      "\n",
      "Created Summary:\n",
      "great\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i keep trying other brands cheaper brands stupid me this ginger is soooo worth the money tender moist and never a let down\n",
      "\n",
      "Actual Summary:\n",
      "simply the best\n",
      "\n",
      "Created Summary:\n",
      "best best\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "works with chicken fish beef or pork fast easy and makes it taste excellent plus buying in bulk is more than 50 off from box stores\n",
      "\n",
      "Actual Summary:\n",
      "awesome stuff\n",
      "\n",
      "Created Summary:\n",
      "good stuff\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "got this for my brother who is on jorge cruise diet and decided to try one for myself it actually tastes pretty good\n",
      "\n",
      "Actual Summary:\n",
      "tastes good\n",
      "\n",
      "Created Summary:\n",
      "tastes good\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "perfect size sea salt for the table or the picnic basket we love it shakes well no clumping and flows freely\n",
      "\n",
      "Actual Summary:\n",
      "great tasting sea salt with iodine\n",
      "\n",
      "Created Summary:\n",
      "the true true\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "these are the best tasting tuna pack they make in my opinion make a great on the go snack and really satisfying with the tomato\n",
      "\n",
      "Actual Summary:\n",
      "tasty\n",
      "\n",
      "Created Summary:\n",
      "addictive\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "this is an great product the taste is great and it works exactly as described superb natural sleep aid amazing\n",
      "\n",
      "Actual Summary:\n",
      "marley s mellow mood lite half tea half lemonade\n",
      "\n",
      "Created Summary:\n",
      "marley ghee\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "we had trouble finding this locally delivery was fast no more hunting up and down the flour aisle at our local grocery stores\n",
      "\n",
      "Actual Summary:\n",
      "yum falafel\n",
      "\n",
      "Created Summary:\n",
      "perfect falafel\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i like these better than the regular altoids but they re even more costly so you d better like them\n",
      "\n",
      "Actual Summary:\n",
      "nice little mints but pricey\n",
      "\n",
      "Created Summary:\n",
      "altoids altoids altoids\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "be careful not to eat too many of them in one day as one of the chemicals in it is the same as that of a laxative lololol\n",
      "\n",
      "Actual Summary:\n",
      "sugarfree\n",
      "\n",
      "Created Summary:\n",
      "okay\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "these mints are really strong and have a great taste they are also a convenient size to carry in a pocket or purse\n",
      "\n",
      "Actual Summary:\n",
      "tasty\n",
      "\n",
      "Created Summary:\n",
      "tasty tasty\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "received as shown my mother in law likes these and walgreens quit carrying them it was great i could get them on amazon\n",
      "\n",
      "Actual Summary:\n",
      "altoids\n",
      "\n",
      "Created Summary:\n",
      "altoids\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "fresh scent and taste easy to eat one tin in few hours i use them as favors and everyone loved them\n",
      "\n",
      "Actual Summary:\n",
      "fresh\n",
      "\n",
      "Created Summary:\n",
      "fresh tasty\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "hey the description says 360 grams that is roughly 13 ounces at under 4 00 per can no way that is the approximate price for a 100 gram can\n",
      "\n",
      "Actual Summary:\n",
      "price can not be correct\n",
      "\n",
      "Created Summary:\n",
      "price price but that\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "this is an excellent tea one of the best i have ever had it is especially great when you prepare it with a samovar\n",
      "\n",
      "Actual Summary:\n",
      "best tea ever\n",
      "\n",
      "Created Summary:\n",
      "excellent\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "herbal additives in this blend destroy real tea taste it is only for people who like herbal taste i don t\n",
      "\n",
      "Actual Summary:\n",
      "not a real tea\n",
      "\n",
      "Created Summary:\n",
      "bland tea tea\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "took me one or two to get used to the pickle taste but now aim hooked and have to keep a bottle on hand\n",
      "\n",
      "Actual Summary:\n",
      "mcclures bloody mary mix\n",
      "\n",
      "Created Summary:\n",
      "mcclures\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "fresh a great way to get a little chocolate in my life without a million calories they taste just like chocolate pudding\n",
      "\n",
      "Actual Summary:\n",
      "omg best chocolate jelly belly\n",
      "\n",
      "Created Summary:\n",
      "omg\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i have tried it out despite the other review and i found that this is exactly the same tea i buy in either russian or italian groceries\n",
      "\n",
      "Actual Summary:\n",
      "this is what you get in the store\n",
      "\n",
      "Created Summary:\n",
      "cheaper fine\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "ahmad tea is an excellent looseleaf tea eith hot or for making iced tea great flavor with no lingering aftertaste\n",
      "\n",
      "Actual Summary:\n",
      "ahmad tea\n",
      "\n",
      "Created Summary:\n",
      "ahmad ahmad\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "if used on a daily basis you will keep your gums healty and between your teeth clean and free of plack\n",
      "\n",
      "Actual Summary:\n",
      "keeps you out of the dentest chair\n",
      "\n",
      "Created Summary:\n",
      "leaking machine\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "five minutes in one tentacle was bitten off ball inside cracked in half not durable enough to be a dog toy disappointed so is the dog\n",
      "\n",
      "Actual Summary:\n",
      "sad outcome\n",
      "\n",
      "Created Summary:\n",
      "sad outcome\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "love this sugar i also get muscavado sugar and they are both great to use in place of regular white sugar recommend\n",
      "\n",
      "Actual Summary:\n",
      "sugar in the raw\n",
      "\n",
      "Created Summary:\n",
      "sugar sugar\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "product is exactly as advertised what a savings at least half the retail price easy to store because of the individual packaging\n",
      "\n",
      "Actual Summary:\n",
      "sugar in the raw\n",
      "\n",
      "Created Summary:\n",
      "product product\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i only use raw sugar it did seem a little smaller than the normal crystals but it is still good will buy again\n",
      "\n",
      "Actual Summary:\n",
      "good product\n",
      "\n",
      "Created Summary:\n",
      "the the\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i really love this sugar and price wise it s much cheaper to buy it here on amazon compared to anywhere else\n",
      "\n",
      "Actual Summary:\n",
      "great\n",
      "\n",
      "Created Summary:\n",
      "great\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "this is one of the best tasting crisps i have ever eaten i only buy it on special occassions because i know i ll eat the whole bag\n",
      "\n",
      "Actual Summary:\n",
      "excellent taste\n",
      "\n",
      "Created Summary:\n",
      "great taste\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i bought these for my husband and he said they are the best energy shots out there he takes one in the mornings and works hard all day good stuff\n",
      "\n",
      "Actual Summary:\n",
      "great energy\n",
      "\n",
      "Created Summary:\n",
      "great\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "you have to try the vanilla tootsie rolls the regular chocolate ones are good but these are the best ever\n",
      "\n",
      "Actual Summary:\n",
      "vanilla tootsie rolls\n",
      "\n",
      "Created Summary:\n",
      "yummy\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "i wouldn t even think of buying this product unless i know the price per unit how can i compare other items online or in a store\n",
      "\n",
      "Actual Summary:\n",
      "what quantity is it\n",
      "\n",
      "Created Summary:\n",
      "price price\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ----------------------------------------------------------------------------------------------------\n",
      "Actual Text:\n",
      "it is hot i love it tasty and a little sweet i usually scoff at salsas labeled as hot but this one lives up to its billing\n",
      "\n",
      "Actual Summary:\n",
      "hot\n",
      "\n",
      "Created Summary:\n",
      "delicious\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# show results\n",
    "sample_results(preds,\n",
    "                                      ind2word,\n",
    "                                      word2ind,\n",
    "                                      converted_summaries[:50],\n",
    "                                      converted_texts[:50])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "j_bcG5CPmw2m"
   },
   "source": [
    "# Conclusion\n",
    "\n",
    "Generally I am really impressed by how well the model works. \n",
    "We only used a limited amount of data, trained it for a limited amount of time and used nearly random hyperparameters and it still delivers good results. \n",
    "\n",
    "However, we are clearly overfitting the training data and the model does not perfectly generalize.\n",
    "Sometimes the summaries the model creates are good, sometimes bad, sometimes they are better than the original ones and sometimes they are just really funny.\n",
    "\n",
    "\n",
    "Therefore it would be really interesting to scale it up and see how it performs. \n",
    "\n",
    "To sum up, I am impressed by seq2seq models, they perform great on many different tasks and I look foward to exploring more possible applications. \n",
    "(speech recognition...)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "Model 3.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
